JUSTICE concerns over social work use of transcription AI

Social worker use of transcription AI may have led to some efficiencies for social workers, but it is impacting the integrity of written documents, with significant risks downstream for family justice.

Recent Ada Lovelace Institute research has identified harmful hallucinations from these technologies in real-world social work cases, including one AI-generated summary which stated a person had expressed suicidal ideation when they had not.

If a family court were to receive such a hallucination in social work evidence, they would be hamstrung to determine it to be as such, since a standard is fast emerging amongst social work data retention policies that these tools are configured to delete the original audio in 30 days, thereby losing an authoritative piece of evidence by which the court could determine what is authentic and what is hallucinated.

JUSTICE is advocating for a systems-approach to arriving at agreed standards for AI tools which will impact the fairness of court proceedings, rather than these standards being decided unilaterally by the commercial vendors. This includes data and audio preservation and auditability standards. This work has started in criminal justice by police and the CPS: their joint operating procedure highlights the importance of retaining original audio when AI is used for transcription in a criminal context.

JUSTICE recommends the right standards for family proceedings – which take into account the need to preserve audio for the purpose of authenticating AI-generated transcripts, but also consider data limitation and practical storage capacity constraints – should be determined through system-wide discussions, including the perspectives of the judiciary, social workers, people affected by care processes, and regulators who establish standards for social work.

Read the full briefing here.

This briefing was edited on 16.09.26 to reflect the following:

1. The majority of social workers spoken to in the Ada Lovelace Institute’s research used either Microsoft Copilot or Beam’s Magic Notes. Paragraph 8 previously just referred to practitioners “using AI” to transcribe. Elsewhere in the blog, Magic Notes was mentioned by name, but Copilot was not. Copilot has therefore been mentioned explicitly to correct any misunderstanding that social workers spoken to only used Magic Notes.
2. The Ada Lovelace Institute research did not specify which AI tool made the harmful hallucination of suicidal ideation. The previous version wrongly attributed this to Magic Notes when no such attribution is made in the research – paragraph 9 has been edited accordingly.
3. A standard is emerging amongst social work use of these tools: they are often configured to automatically delete the original audio in 30 days. However the previous version of the blog suggested they were “designed” this way. Beam have informed us that customers are able to configure the tool to longer periods of retention if they choose: 30 days is a customer choice, not a fixed design feature. Paragraphs 15 and 20 have been edited to correct any misunderstanding.