In a case that made headlines earlier this year, an attorney with Dudley DeBosier was cited by a Baton Rouge state court judge for including a fabricated quote in a legal pleading drafted with the help of an artificial intelligence tool.
In a written apology to the court, the attorney said he felt “surprise, confusion, shame and regret” over the incident, explaining that the firm had recently begun using a new AI platform for lawyers called EVE.
“I cannot be sure if the mistake involved the EVE AI software or if this was my personal mistake,” Ross LeBlanc wrote in his March apology to 19th Judicial District Court Judge William Jorden. “Either way, the mistake is ultimately mine.”
The incident was one of more than a dozen in Louisiana state and federal courts since last August in which lawyers have been fined or sanctioned for citing inaccurate or fabricated quotes — called hallucinations — in legal pleadings that have been drafted with the help of AI.
It’s a problem that is causing headaches in the legal system and reshaping the way lawyers and courts do some of the most basic functions of their job.
“Courts are getting increasingly exhausted, saying it’s not an excuse anymore for lawyers to say they didn’t know chatbots could make up cases,” said attorney Andy Lee, a partner with Jones Walker who helps companies deal with legal issues around the use of AI. “But lawyers are still doing it.”
Louisiana lawyers are not alone. More than 1,700 incidents of hallucinated content in legal briefs have been flagged by courts around the world since 2022, the year that OpenAI introduced the first generative AI platform, ChatGPT.
More than 70% of those incidents occurred in the U.S. And the numbers are growing fast, according to Damien Charlotin, a researcher and writer who maintains a database of AI hallucination cases.
“Pre-2025, there were two or three hallucination incidents a month,” he wrote in a blog post on his website in November. “Now, that’s the daily average.”
He attributed the surge to the increased availability of large language models like ChatGPT and Claude and to greater public awareness of AI tools.
‘Made up more stuff’
The technology has already touched all types of law in Louisiana, from billboard attorneys and bankruptcy lawyers to old-line firms that specialize in real estate deals and government lawyers.
In March, a federal district judge in the U.S. Eastern District of Louisiana fined two assistant city attorneys in New Orleans for including nine AI hallucinations in pleadings they filed in a case alleging civil rights violations, court records show.
The city did not respond to a request for comment.
In April, a federal bankruptcy judge in Lafayette fined an attorney $2,750 for filing “false and misleading pleadings” that she admitted to drafting with help from ChatGPT. According to court records, federal bankruptcy Judge John Kolwe, for the Western District of Louisiana, said he hoped the fine would “serve as a warning to all attorneys and persons practicing in this court,” adding that future fines for AI hallucinations would not be so “lenient.”
In May, a federal district judge in Alexandria sanctioned a personal injury attorney for using hallucinated quotes in a legal brief generated by the AI platform Claude. Court records from the U.S. District Court for the Western District of Louisiana show that a law clerk initially caught the faulty quote before the brief was filed and notified the attorney, who then “entrusted” Claude to fix it.
“Instead, Claude just made up more stuff,” District Judge Jerry Edwards said in court documents.
The lawyer, Paul Wilkins, also with Dudley DeBosier, apologized to the court and vowed that a human clerk would double check any future pleadings he filed with the court. Still, Edwards fined him $1,000, saying, “Ignorance of the risks of AI usage is no longer an excuse.”
Wilkins did not respond to a request seeking comment. LeBlanc, the Dudley DeBosier attorney cited in the Baton Rouge case, declined to comment.
Dudley DeBosier founding partner Chad Dudley acknowledged the firm has had trouble with AI platforms and is working to address them.
“We are always looking for ways to use technology to get better results for our clients and have successfully implemented the use of AI in a number of ways over the past 18 months,” he said. “We did have issues with its use in case research and case citations. We take those issues very seriously and immediately implemented an AI usage policy for our attorneys.”
Fear factor
Experts say the use of generative AI in the legal field is inevitable because the technology is advancing too quickly and is already too widespread to ignore.
For all its faults, Lee said, AI can be “extremely useful” for brainstorming, distilling hundreds of pages of material and transcripts and coming up with novel arguments in legal proceedings.
Problems arise when attorneys or their staff become overly reliant on the technology without checking the cites and quotes it pulls up, according to Loyola University law professor Dane Ciolino.
Just as dangerous, he said, is the tendency to ignore the technology for fear of using it incorrectly and getting in trouble.
“I see so many lawyers who read the reports of the hallucinated case sanctions levied against other lawyers and say, ‘I’m going to stay away’ and that is not acceptable,” Ciolino said. “This is a very powerful tool that lawyers not only can and should but must be using for the benefits of their clients.”
The key to successfully using AI tools in a legal practice is to understand the way it works and become well versed in the limitations and the potential advantages of the technology.
“It is not a law library,” Lee said. “It is a predictive mechanism and a people pleaser, set up to be a sycophant to its user.”
That’s why phantom quotes or hallucinations pop up in pleadings, Lee said. The technology is trying to support the legal argument in the most effective way possible, even if that means fabricating something.
Ciolino said it’s the user’s responsibility to safeguard against that possibility by carefully prompting the platform: limiting which sources it can research, telling it where to look and directing it to assess its level of certainty in the sources.
Lee advises telling AI “not to suck up but to be critical and find where your argument has holes or problems and to tell you what the other side is going to attack.”
It’s also critical, he said, that attorneys read everything for themselves, essentially double-checking AI’s work.
“There are safeguards for lawyers who are using it responsibly,” Lee said. “But they are not fail-safe, and lawyers get busy.”
Brave new world
The controversy generated by the recent surge in AI hallucinations in the legal field underscores larger and more profound shifts that the technology is already foretelling.
AI, used correctly, can also make firms more efficient. Ultimately that may mean that firms will be able to do more with fewer researchers, clerks, paralegals and attorneys.
“No doubt it is going to affect how you staff law firms,” Lee said. “The lawyers who resist and do not adapt are going to lose business as a result. But it doesn’t mean the business goes away. It will go to those who are adapting.”
