A judge in California has imposed sanctions to law firms that relied on AI for case research, resulting in an error-filled brief.

Listen to a lively discussion about using GenAI for legal briefs:

In the case of Lacey v. State Farm, Judge Michael Wilner (serving as Special Master in the case) took the two law firms representing the plaintiff to task for what he described as “bogus AI-generated research.”

As recounted in detail in orders I issued on April 15 and 20 (attached to the Appendix to this order), Plaintiff’s supplemental brief contained numerous false, inaccurate, and misleading legal citations and quotations. According to my after-the-fact review—and supported by the candid declarations of Plaintiff’s lawyers—approximately nine of the 27 legal citations in the ten-page brief were incorrect in some way. At least two of the authorities cited do not exist at all. Additionally, several quotations attributed to the cited judicial opinions were phony and did not accurately represent those materials. The lawyers’ declarations ultimately made clear that the source of this problem was the inappropriate use of, and reliance on, AI tools.

How It Happened

Judge Wilner then goes on to describe the sequence of events that happened, wherein an attorney (Mr. Copeland) at one of the two firms (Ellis George) representing the plaintiff “used various AI tools to generated an ‘outline’ for the supplemental brief. That document contained the problematic legal research.”

When the attorney sent the outline to the second firm (K&L Gates), it was included in the brief, with not attorneys or staff cite-checking or reviewing it for accuracy. Judge Wilner said the attorneys at the second firm, K&L Gates, were unaware the document contained AI-generated research.

A further wrinkle. During my initial review of Plaintiff’s brief, I was unable to confirm the accuracy of two of the authorities that hte lawyers cited. I emailed the lawyers shortly after receiving the brief to have them address this anomaly. Later that day, K&L Gates re-submitted the brief without the two incorrect citations—but with remaining AI-generated problems in the body of the text. An associate attorney sent me an innocuous e-mail thanking me for catching the two errors that were “inadvertently included” in the brief, and confirming that the citations in the Revised Brief had been “addressed and updated.”

I didn’t discover that Plaintiff’s layers used AI—and re-submitted the brief with considerably more made-up citations and quotations beyond the two initial errors—until I issued a later OSC soliciting a more detailed explanation. The lawyers’ sworn statements and subsequent submission of the actual AI-generated “outline” made clear the series of events that led to the false filings. The declarations also included profuse apologies and honest admissions of fault.

‘No Reasonably Competent Attorney Should’ Rely On AI

Judge Wilner then goes on to cite a number of legal rules pertaining to an attorneys’ responsibility to provider documents and briefs that are accurate, to the best of their ability. He also laments the fact that courts are increasingly being forced to deal with attorneys improperly using AI in their submissions. He ultimately concludes that the attorneys acted in bad faith, and makes the point that competent attorneys should not be using AI in this manner.

I conclude that the lawyers involved in filing the Original and Revised Briefs collectively acted in a manner that was tantamount to bad faith. Fink, 239 F.3d at 994. The initial, undisclosed use of AI products to generate the first draft of the brief was flat-out wrong. Even with recent advances, no reasonably competent attorney should out-source research and writing to this technology—particularly without any attempt to verify the accuracy of that material. And sending that material to other lawyers without disclosing its sketchy AI origins realistically put those professionals in harm’s way. Mr. Copeland candidly admitted that his is what happened, and is unreservedly remorseful about it.

Despite finding that Mr. Copeland was primarily responsible for creating the situation, Judge Wilner had plenty to say about K&L Gates’ lack of due diligence and the role it played.

Yet, the conduct of the lawyers at K&L Gates is also deeply troubling. They failed to check the validity of the research sent to them. As a result, the fake information found its way into the Original Brief that I read. That’s bad. But, when I contacted them and let them know about my concerns regarding a portion of their research, the lawyers’ solution was to excise the phony material and submit the Revised Brief—still containing a half-dozen AI errors. Further, even though the lawyers were on notice of a significant problem with the legal research (as flagged by the brief’s recipient: the Special Master), there was no disclosure to me about the use of AI. Instead, the e-mail transmitting the new brief merely suggested an inadvertent production error, not improper reliance on technology. Translation: they had the information and the chance to fix this problem, but didn’t take it.

I therefore conclude that (a) the initial undisclosed use AI, (b) the failure to cite-check the Original Brief, and (perhaps most egregiously), (c) the re-submission of the defective Revised Brief without adequate disclosure of the use of AI, taken together, demonstrate reckless conduct with the improper purpose of trying to influence my analysis of the disputed privilege issues. The Ellis George and K&L Gates firms had adequate opportunities—before and after their error had been brought to their attention—to stop this from happening. Their failure to do so justifies measured sanctions under these circumstances.

Conclusion and Sanctions

Judge Wilner issued his final conclusion, including his decision to levy sanctions.

A final note. Directly put, Plaintiff’s use of AI affirmatively misled me. I read their brief, was persuaded (or at least intrigued) by the authorities that they cited, and looked up the decisions to learn more about them—only to find that they didn’t exist. That’s scary. It almost led to the scarier outcome (from my perspective) of including those bogus materials in a judicial order. Strong deterrence is needed to make sure that attorneys don’t succumb to the easy shortcut.

for these reasons, Plaintiff’s supplemental briefs are struck, and no further discovery relief will be granted on the disputed privilege issue. Additionally, Plaintiff’s law firms are ordered to (jointly and severally) to pay compensation to the defense in the aggregate amount of $31,000.

A Damning Indictment of AI Reliability

Judge Wilner’s experience with AI is, unfortunately, all too common. Hallucinations continue to be a major problem for AI models, with even recent models continue to struggle. In fact, some of OpenAI’s most advanced models actually hallucinate more than previous ones.

AI firms are pushing for widespread adoption of their AI models and, even more concerning, some lawmakers want to prevent states and agencies from regulating AI. As a result, some users falsely believe AI is ready to tackle any task its given, and can be trusted to do so accurately and reliably.

Incident after incident, however, shows that AI still has a long way to go before it can be fully trusted to handle even the most basic task—such as researching legal cases without making ones up.