Two lawyers rapped over ‘entirely fictitious’ AI-generated citations submitted to court

[SINGAPORE] Lawyers from two local law firms were rapped by the High Court for citing “entirely fictitious” cases hallucinated by artificial intelligence (AI) tools, in the Republic’s second such case to be unearthed.

In his judgment delivered on Monday (Nov 3), Justice S Mohan addressed two non-existent legal cases that appeared in closing submissions filed in a loan recovery dispute.

“What was most troubling was the defendants’ citation of two authorities… of their closing submissions which were entirely fictitious,” said the judge.

Justice Mohan noted that there were “reasonable grounds to suspect that the fictitious authorities were likely to have been generated by AI tools used in the preparation of the defendants’ closing submissions”.

He added: “It is now well-known that AI tools, when utilised in the drafting of legal submissions, carry the risk of ‘hallucinating’ plausible sounding but entirely fabricated legal ‘authorities’.”

The cases were surfaced when the opposing counsel from David Lim & Partners flagged in their reply submissions that the cited case could not be located anywhere in legal databases.

Counsel for the defendants, Goh Peck San from P S Goh & Co, was then asked by the court whether the paragraphs referencing the two cases had been generated by an AI tool.

Goh subsequently admitted that the citations were fictitious. He explained, however, that he had engaged a fellow lawyer, Amarjit Singh Sidhu from Amarjit Sidhu Law, to assist with research and did not know that an AI tool had been used.

Sidhu later told the court that it was an “honest oversight” that resulted in the fictitious cases being cited in the defendant’s submissions. Goh and Sidhu have not confirmed which AI tool was used to generate the fictitious cases.

SEE ALSO

Breaches of AI policy could be a sackable offence at some Singapore law firms

I’ve been deepfaked – and it can happen to you

Justice Mohan reserved judgment on what consequences should follow from the defendants’ citation of fictitious cases, and will issue further directions in a subsequent judgment, pending additional detail from Goh and Sidhu.

This is the second reported case of AI-generated fake citations in Singapore court submissions. In an earlier case in October, the High Court ordered a lawyer to personally pay S$800 in costs for citing a hallucinated case.

Following that case, some local law firms told The Business Times that individuals who breach the firms’ AI policies could face disciplinary action, including being sacked.