Police forces across England and Wales have been ordered to halt the use of artificial intelligence tools in court statements and other criminal justice tasks after concerns emerged that inaccurate AI-generated content could compromise legal proceedings.
The intervention comes amid growing scrutiny of generative AI systems and their tendency to produce convincing but false information, known as ‘hallucinations.’ Officials say any technology used within the justice system must meet exceptionally high standards of accuracy before it can be deployed in legal processes.
Police Forces Told to Pause AI Use
According to the Financial Times, several police forces had begun using commercially available AI tools to assist with court statements and other criminal justice work before comprehensive assessments had been completed.
Alex Murray, head of the newly established Police.AI centre, said he had intervened in some cases to stop the rollout.
‘We need to slow it down a bit,’ Murray said, adding that technologies used within the criminal justice system must achieve a standard of accuracy that is ‘beyond reasonable doubt.’
The directive affects the use of AI in preparing court-related documents and evidence-handling processes, areas where factual accuracy is critical to ensuring fair legal outcomes. Authorities fear that even minor inaccuracies generated by AI could contaminate evidence, mislead courts or undermine confidence in criminal proceedings.
Concerns Over AI ‘Hallucinations’
The warning follows instances in which AI systems produced inaccurate information while being used in policing contexts.
One case cited by the Financial Times involved the use of Microsoft’s Copilot tool, which reportedly generated false information about a football fixture while contributing to a police intelligence dossier. The incident heightened concerns that AI-generated errors could find their way into official documents if not properly checked.
Experts have repeatedly warned that large language models can generate responses that appear authoritative despite containing factual errors.
The issue has already affected legal systems in the UK and elsewhere. Courts and regulators have recently dealt with multiple cases involving lawyers who submitted AI-generated legal citations that later proved to be entirely fictitious.
Balancing Innovation With Legal Integrity
The decision does not represent a rejection of AI within policing. Instead, officials say it reflects the need for stronger safeguards before the technology is integrated into sensitive legal processes.
‘All forces have got a good policy on the use of Copilot,’ according to Murray. ‘All forces will have a policy that says, “Check everything that it produces”.’
Police.AI was established as part of broader government efforts to improve efficiency across policing. The initiative has been backed by funding intended to help forces explore new technologies while maintaining public trust and operational standards.
Murray has argued that AI still has significant potential in areas where risks can be more effectively managed. These include analysing CCTV footage, processing large volumes of digital evidence, and helping officers review distressing material in child abuse investigations. Such applications could reduce workloads and limit officers’ exposure to traumatic content.
Growing Scrutiny Across the Legal Sector
The police directive comes at a time when AI use across the legal profession is facing increasing examination.
Recent cases involving solicitors, barristers and law firms have highlighted the dangers of relying on AI-generated material without proper oversight. In some instances, courts have criticised legal professionals for submitting fabricated case references produced by AI tools. Regulators have warned that responsibility for accuracy ultimately remains with the individual submitting the information.
Legal experts generally agree that AI can deliver substantial efficiency gains, but caution that human review remains essential wherever evidence, legal arguments or judicial decisions are involved.
Safeguards Expected Before Wider Adoption
For now, police forces have been instructed to pause certain AI applications until they undergo rigorous assessment and meet required standards.
Officials say the move is intended to ensure that technological innovation does not come at the expense of fairness, transparency or legal accuracy.
As law enforcement agencies increasingly explore AI-powered tools, the episode serves as a reminder that while automation may accelerate administrative tasks, accountability for the information presented in court ultimately rests with humans.
Leave a comment