AI Hallucination Check: Real Cases and a Check-Before-You-Send List
AI tools sometimes invent things: cases, quotes, books, policies, figures. They do it fluently, which is why it gets past busy people. Below are real, documented AI hallucination cases with sources, what each one teaches a small business, and a checklist to run before AI-drafted work reaches a client.
What is an AI hallucination?
An AI hallucination is output that's false or made up but presented as fact. Generative AI produces likely-sounding text; it doesn't look things up unless the tool is built to, and even then it can misread or misquote. Examples of AI hallucination include citations to court cases that don't exist, quotes nobody said and products or policies that were never offered. The risk isn't that AI is always wrong. It's that it's wrong in the same confident tone it uses when it's right.
Real AI hallucination cases and what they teach
Fake court cases in a legal filing (Mata v. Avianca, 2023)
Lawyers filed a brief citing judicial opinions that ChatGPT had invented, then kept standing by them after the court questioned whether they existed. The judge sanctioned the two attorneys and their firm, with a $5,000 penalty.
Lesson: Asking the AI whether its sources are real is not a check. Look the source up yourself.
Sources: Opinion and order on sanctions · ACC analysis
A chatbot invented a refund policy (Moffatt v. Air Canada, 2024)
Air Canada's website chatbot told a grieving customer he could claim a bereavement fare discount after travel. The airline's actual policy said otherwise. A British Columbia tribunal found the airline liable for negligent misrepresentation and ordered it to compensate him.
Lesson: The business owns what its chatbot says. Linking to the correct policy elsewhere didn't excuse the wrong answer.
Sources: American Bar Association · Dentons
A consultancy report with fabricated references (Deloitte Australia, 2025)
A report Deloitte Australia produced for a federal government department contained apparent AI-generated errors, including a fabricated quote from a court judgment and references to nonexistent research papers. Deloitte agreed to partially refund the A$440,000 fee, and a revised version was published.
Lesson: Big firms aren't immune. Any document with citations needs every reference opened and checked.
Sources: AP News
A newspaper reading list of books that don't exist (2025)
A syndicated summer supplement printed in the Chicago Sun-Times and The Philadelphia Inquirer included a reading list in which many of the books didn't exist, attributed to real authors. The freelance writer said he used AI and didn't fact-check it; the content distributor ended its relationship with him.
Lesson: Content from a supplier is still your content once you publish it. Check outsourced work too.
Sources: Chicago Sun-Times · AP News
Hundreds more in the courts
Researcher Damien Charlotin keeps a public AI Hallucination Cases database of court and tribunal decisions where a party relied on hallucinated content, typically fake citations. When we checked in October 2026 it listed more than 2,000 cases. Legal filings are simply where hallucinations get caught on the record; the same errors happen in proposals, reports and emails, where nobody may notice.
The check-before-you-send checklist
Run this on any AI-drafted work that leaves the firm. Weighted items count double. The maximum is 14.
Can an AI hallucination checker or detector replace this?
Tools sold as AI hallucination checkers or detectors can help flag claims that need checking, and some AI tools now show their sources. But a tool checking a tool still leaves the question in Mata v. Avianca unanswered: did a person open the source? For anything client-facing, treat automated checks as a first pass and keep the human read. Count that review time when you work out whether AI is paying off; our AI ROI calculator includes it.
How to reduce hallucinations at the source
- Give it your material. Ask AI to work from documents you supply, not from memory.
- Ask for sources, then check them. A listed source is a lead to verify, not proof.
- Keep AI away from final facts. Use it to draft and structure; take figures and citations from the originals.
- Make one person accountable. Each AI use needs a named owner who checks output. Our AI governance checklist sets that up.
- Teach the habit. Checking output is one of the five areas in the AI literacy quiz.
For the wider list of where small firms go wrong with AI, see AI mistakes businesses make.
Frequently asked questions
What is an AI hallucination?
Output from an AI tool that's false or made up but presented as fact, such as invented citations, quotes, policies or figures.
What are famous AI hallucination cases?
Mata v. Avianca (fake court cases in a filing), Moffatt v. Air Canada (a chatbot's invented fare policy), the Deloitte Australia government report with fabricated references, and the 2025 newspaper reading list of nonexistent books.
Who is responsible for AI hallucinations?
In the cases above, responsibility fell on the people and businesses who relied on or published the output, not the AI tool.
Are AI hallucination detectors reliable?
They can help flag claims to check, but they don't replace a person opening the sources for anything client-facing.
How do I stop AI from making things up?
Give it your own source material, ask for sources and check them, take final facts from originals and have a named person review output before it goes out.
Do paid AI tools hallucinate less?
Some tools add source links or work from your documents, which helps. None removes the need to check.