AI Hallucinations in Court: The New Line Lawyers Cannot Cross

Overview

The legal profession is navigating an unprecedented crisis as artificial intelligence tools increasingly populate court filings with fabricated precedents and non-existent statutes. There have been at least 2,000 legal cases involving AI hallucinations, creating a backlog of ethical and procedural challenges for courts worldwide. The United States accounts for the vast majority of these incidents, with 1,473 cases, followed by Canada with 229 and Australia with 112.

Using AI is fine; signing unread output is not. Courts are converging on verification, not disclosure. Sanctions have moved from fines to licences.

This trend has forced a reevaluation of professional responsibility. While some attorneys argue for a complete ban on large language models in legal work, others see them as vital for expanding access to justice. The emerging consensus is straightforward: if you file it, you own it. As accountability mechanisms strengthen, stripping AI from legal practice entirely appears to be an overreaction that would ignore its potential utility.

From Fines to Disbarment: A Shift in Sanctions

Historically, penalties for citing fake cases might have resulted in mere reprimands or small fines. Today, the consequences are severe and often career-ending. In December 2025, a federal judge in Oregon imposed $110,000 in fees and fines on lawyers who submitted 15 fake cases and eight fake quotations. This remains the record total for such violations.

By April 2026, the Nebraska Supreme Court suspended an attorney after 57 of 63 citations in their brief were found to be defective. The court noted this was the first instance where AI use had been explicitly denied by the filer. Similarly, in Withers v. City of Aberdeen in June 2026, both sides of a lawsuit filed hallucinated citations. The result was the removal of all four lawyers involved; the drafters were barred from the district for two years, while non-drafting co-signers faced fines and mandatory continuing legal education.

In Withers, two lawyers who claimed they never use AI were still removed and fined, establishing that co-signing counts as using.

Internationally, the trend continues. In September 2026, the Solicitors Regulation Authority in England struck off a foreign lawyer for relying on AI-fabricated authorities in his own disciplinary defense. In the United States, courts are notably tougher on licensed attorneys than on pro se litigants, who typically receive fee orders or filing restrictions rather than license sanctions.

Judicial Missteps and Institutional Responses

The problem is not confined to practicing attorneys. The bench and the government have also stumbled. In October 2025, Judges Wingate and Neals withdrew error-riddled orders after disclosing that their staff had used Perplexity and ChatGPT. Their admission came only after Senate questions, highlighting a lack of transparency.

Later that year, Chief Judge Jarbou rebuked the Justice Department for an apparently AI-generated Sixth Circuit citation but did not sanction the agency. In a more direct intervention, Judge Cooper enjoined HHS grant changes in September 2026, ruling that they rested on studies that do not exist.

In response to these institutional failures, Iowa enacted a new judicial rule in 2027. This rule explicitly prohibits judges from delegating decision-making responsibility or deferring to AI-generated work product. It represents a significant step toward ensuring that judicial reasoning remains human-driven and accountable.

Diverging Regulatory Frameworks

States are currently grappling with how to regulate AI in legal filings, resulting in a patchwork of rules. New York, which implemented Part 161 in June 2026, permits AI use but requires signers to certify a review for fabricated material. Notably, it does not require disclosure of AI use. Florida adopted a similar approach in June 2026, replacing circuit-level disclosure orders with a rule that never mentions AI by name but mandates certification that authorities exist and are accurately cited.

California is taking a stricter path with SB 574, effective January 2027. This law requires disclosure of AI use to the court, mandates that attorneys personally read and verify every citation, and prohibits inputting confidential data into public AI tools. Additionally, arbitrators may not delegate decisions to AI under this statute.

The proposed Federal Rules of Evidence amendment to Rule 707, revised in May 2026, addresses AI output offered as evidence. Without an expert witness, such output must meet the reliability standards of Rule 702. Meanwhile, the Tenth Circuit has proposed rules requiring AI-assisted filings to certify independent human verification.

Drawing the Line on Acceptable Use

Legal scholars and ethicists are now defining the boundaries of acceptable AI integration. The settled zone allows drafting, editing, summarizing, and research leads with any tool, provided a human checks them. However, significant contestation remains over whether to disclose AI use, how to handle client data in public tools, and the appropriate level of punishment for errors.

The line is clear: citing an authority or quotation you have not read, at any level of intent, is crossing the line.

Crossing the line also includes inventing facts or testimony and hiding prompts intended for AI readers. The Arizona Court of Appeals reinforced this boundary in State v. Horcasitas in September 2026, vacating a sentence after the prosecution introduced an AI-generated video of a deceased victim "speaking" at sentencing. The court deemed the evidence unreliable, marking another precedent where technology overreach led to procedural reversal.

As the dust settles, the legal community is moving toward a framework where verification trumps disclosure. The focus is no longer on whether AI was used, but on whether the human signer took responsibility for the accuracy of the work product. This shift ensures that while AI can assist in the legwork of law, it cannot absolve lawyers of their duty to the court.