JUSTICE concerns over social work use of transcription AI

Social worker use of transcription AI may have led to some efficiencies for social workers, but it is impacting the integrity of written documents, with significant risks downstream for family justice.

Recent Ada Lovelace Institute research has identified harmful hallucinations from these technologies in real-world social work cases, including one AI-generated summary which stated a person had expressed suicidal ideation when they had not.

If a family court were to receive such a hallucination in social work evidence, they would be hamstrung to determine it to be as such, since these tools delete the original audio in 30 days as standard, thereby losing an authoritative piece of evidence by which the court could determine what is authentic and what is hallucinated.

JUSTICE is advocating for a systems-approach to arriving at agreed standards for AI tools which will impact the fairness of court proceedings, rather than these standards being decided unilaterally by the commercial vendors. This includes data and audio preservation and auditability standards. This work has started in criminal justice by police and the CPS: their joint operating procedure highlights the importance of retaining original audio when AI is used for transcription in a criminal context.

JUSTICE recommends the right standards for family proceedings – which take into account the need to preserve audio for the purpose of authenticating AI-generated transcripts, but also consider data limitation and practical storage capacity constraints – should be determined through system-wide discussions, including the perspectives of the judiciary, social workers, people affected by care processes, and regulators who establish standards for social work.

Read the full briefing here.