A senior judge has raised concerns about the dangers of relying on AI for court preparation following the discovery of references to fictitious legal cases in several documents.
Mr Justice Sweeting voiced these concerns while rejecting the appeals of two individuals contesting their extradition to Romania for crimes committed there.
During the High Court proceedings, the Crown Prosecution Service (CPS) opposed the appeals. However, it was revealed at the end of the hearing that two legal references cited by the CPS were fabricated by an AI system and did not have any real basis.
The judge highlighted that these erroneous references were included in another document and only came to light when brought to the attention of the CPS before a February hearing.
The CPS acknowledged the mistake, expressing regret and clarifying that there was no intention to deceive the court. They attributed the error to a lack of verification by the reviewing lawyer before submitting the documents.
In a letter in March, the CPS admitted that the inaccuracies stemmed from the use of artificial intelligence and underscored the necessity of thorough human oversight in legal submissions.
Mr Justice Sweeting mentioned that the CPS conducted an internal review of the incident and examined 78 other cases associated with the same lawyer, finding no similar issues and assuring minimal risk of recurrence.
Regarding the impact on the appeal, the judge confirmed that the errors were rectified before the hearing, ensuring they did not affect the arguments or the court’s decision. He accepted the CPS’s apology and assurance of no deliberate attempt to mislead.
Emphasizing the growing role of AI in legal processes, the judge noted the importance of proper oversight to mitigate risks, especially in legal research.
A CPS spokesperson clarified that they do not use AI in legal decision-making or charge determinations, emphasizing their proactive measures to rectify the error and prevent future occurrences, as validated by the judge’s acknowledgment of the error’s minimal impact on the court’s judgment.
