AI hallucination is when a large language model (LLM) generates text that is factually incorrect, fabricated, or contradicts known reality — stated with the same fluent confidence as accurate information. Examples: citations to papers that don't exist, statistics that were never published, historical "facts" that didn't happen, code that references functions that don't exist. Hallucination is not a bug in individual models; it's an inherent feature of how LLMs work — they generate statistically likely text, not verified truth.
An AI hallucination is a model being confidently wrong.
Where the Term "Hallucination" Comes From
The term "hallucination" in the context of AI language models was adopted from cognitive science — where hallucination refers to perceiving something that isn't there. Applied to AI, it describes the analogous failure: generating text about something that isn't real.
The problem was recognized early in neural language model research. Ji et al. (2023, "Survey of Hallucination in Natural Language Generation," ACM Computing Surveys) catalogued hallucination across NLP tasks including machine translation, summarization, dialogue, and question answering. The authors distinguished:
- Intrinsic hallucination: Output contradicts the input or source material
- Extrinsic hallucination: Output cannot be verified from the input — it introduces information not present in the source
With the rise of large language models and their commercial deployment (ChatGPT, Bard, Claude), AI hallucination moved from a research concern to a public one — high-profile examples included fabricated legal citations submitted in actual court filings, fake scientific references in medical writing, and AI-generated news articles containing false attributions.
Why LLMs Hallucinate
Understanding why LLMs hallucinate requires understanding what they actually do:
LLMs predict probable text, not true text.
An LLM generates tokens (words or sub-words) one at a time, each chosen based on the probability distribution over all possible next tokens given the preceding context. The model learned these probabilities by training on massive text corpora. It optimizes for generating fluent, contextually appropriate text — not for verifying factual accuracy.
When asked "What year did Einstein win the Nobel Prize?", the model generates text that is statistically consistent with what "Nobel Prize + Einstein" text looks like in its training data. If that distribution is well-represented, the answer is likely correct. If it's poorly represented or ambiguous, the model generates the most probable-sounding answer — which may be wrong.
The model doesn't know what it doesn't know.
A critical limitation: LLMs have no reliable internal signal for "I'm uncertain about this." They generate text with confidence regardless of whether the underlying information is strongly represented in training or not. Some models have been trained to express uncertainty more often, but this is an imperfect calibration, not a reliable signal.
Long-tail information is highly susceptible.
Well-covered facts (Einstein's 1921 Nobel Prize) are well-represented in training data and typically accurate. Specific, niche facts (the publication date of a minor journal article, the middle name of a historical minor figure) are poorly represented — the model generates plausible-sounding values rather than admitting it doesn't know.
Types of Hallucination
Factual hallucination:
The most common type — incorrect facts stated as true. "The Eiffel Tower was built in 1887" (it was completed in 1889). "Einstein won the Nobel Prize in 1921 for his theory of relativity" (it was for the photoelectric effect).
Citation fabrication:
LLMs asked to provide sources often generate plausible-sounding paper titles, author names, journals, and years that don't correspond to real publications. This is particularly dangerous in academic and professional contexts.
Confabulation in context:
When given a source document (RAG context, pasted text) and asked to answer questions about it, LLMs may generate answers that go beyond or contradict the source — "filling in" with training knowledge or generating plausible-sounding additional detail.
Entity hallucination:
Inventing specific details about real entities: a biographer who doesn't exist, a law case that was never decided, a product feature that doesn't exist.
Consistency hallucination:
Making internally inconsistent claims within a single response — saying X in paragraph 2 and not-X in paragraph 4.
A Worked Example
A lawyer uses an AI assistant to research precedents for a brief.
Prompt: "Find legal cases supporting the argument that employers cannot require non-disclosure agreements in wrongful termination settlements."
AI hallucination response: "Yes, there are several relevant cases. See Johnson v. Apex Communications (2019, 2nd Circuit), which held that NDAs in wrongful termination settlements violate public policy when they prevent employees from reporting illegal activity. Also Harrison v. Meridian Health Corp. (2021, 9th Circuit), which..."
The reality: These cases don't exist. The model generated plausible-sounding case names, circuit courts, years, and legal holdings that match the pattern of real legal citations but refer to nothing real.
The consequence: A lawyer who cited these cases in a brief without verification would file fabricated citations — as happened in the actual 2023 case of Mata v. Avianca, where a lawyer submitted a ChatGPT-generated brief citing multiple nonexistent cases, resulting in sanctions.
How to Detect and Reduce Hallucination
For developers:
Retrieval-Augmented Generation (RAG):
Instead of asking the LLM to recall from training, retrieve relevant documents and ask the model to answer based on those documents. Significantly reduces factual hallucination by grounding responses in verified sources. Not a complete solution — models can still confabulate beyond provided context.
Uncertainty elicitation:
Prompt the model to express uncertainty: "Answer the question and indicate your confidence level (low/medium/high) and whether the answer could be verified from public sources." Models with calibrated uncertainty training (like Claude's constitutional AI approach) can be prompted to say "I'm not certain about this."
Self-consistency checking:
Ask the same question multiple times with slight variations. If answers are inconsistent, treat outputs with lower confidence.
Factual verification pipeline:
For production applications where factual accuracy is critical, add a verification step: extract specific claims from LLM output, check each against a verified knowledge base or search results.
For knowledge workers and users:
Verify before using:
Any specific fact, statistic, citation, date, or proper noun generated by an LLM should be independently verified before being used in any consequential context (publications, legal documents, medical advice, business decisions).
Treat citations with extreme suspicion:
AI-generated citations are particularly unreliable. Always independently verify that a cited paper, case, or source exists and says what the model claims it says.
Use RAG-grounded tools for factual tasks:
Perplexity AI, Bing AI, and Google's AI search retrieve sources before generating answers and provide verifiable citations. These significantly reduce (but don't eliminate) hallucination compared to base LLMs.
Don't use LLMs for high-stakes factual recall:
For legal research, medical facts, financial regulations — use authoritative sources. LLMs are valuable for drafting, explaining, summarizing, and generating, not for serving as the authoritative source of factual truth.
Hallucination Rates: What Research Shows
Hallucination rates vary significantly by task, model, and measurement methodology:
| Task | Approximate hallucination rate | Source |
|---|
| Medical question answering | 15-40% depending on domain | Umapathi et al. (2023) |
| Summarization with sources | 5-30% of summaries contain unsupported claims | Maynez et al. (2020) |
| Citation generation | Very high (70%+ of generated citations are incorrect) | Multiple studies |
| Common factual queries | Lower (2-10% for well-covered facts) | Varies by model |
These rates vary dramatically by model, version, and task. Frontier models (GPT-4, Claude 3.5, Gemini 1.5) hallucinate less than their predecessors, but no current model has eliminated the problem.
Will Hallucination Be Solved?
Progress is real but complete elimination is unlikely with current architectures. Reasons:
- LLMs are trained to generate fluent text, not verified facts — there's no inherent mechanism to distinguish known from generated information.
- The long tail of specific facts will always have coverage gaps in training data.
- Confabulation beyond provided context appears to be difficult to fully eliminate.
Mitigation approaches (RAG grounding, verification layers, uncertainty calibration, tool use for factual lookup) are reducing hallucination rates significantly for production applications. But for the foreseeable future, treating LLM outputs as drafts requiring verification rather than authoritative sources is the appropriate working assumption.
Related Concepts
Large language model (LLM): The system that hallucinate — understanding why LLMs work the way they do explains why hallucination happens.
RAG (Retrieval-Augmented Generation): The primary technical mitigation — grounding LLM responses in retrieved documents reduces hallucination.
Prompt engineering: Prompting techniques (explicit uncertainty requests, RAG integration, chain-of-thought) can reduce hallucination rates.
Calibration: The property of a model expressing uncertainty in proportion to its actual accuracy — poorly calibrated models hallucinate without indicating uncertainty.
Frequently Asked Questions
Is hallucination the same as lying?
No — lying implies intentional deception, which requires intent. LLMs don't have intent; they generate statistically probable text. "Confabulation" (confident generation of information that feels consistent) is a more precise term from cognitive science. The model isn't lying; it's generating fluent text that happens to be false.
Why don't AI companies fix this?
They're working on it — RAG grounding, RLHF training for uncertainty, tool-use verification, constitutional AI approaches all reduce hallucination rates. But eliminating it entirely would require fundamental changes to how LLMs work (or architectural additions like mandatory factual verification). The problem is partly inherent to the generative approach.
How do I know when an AI is hallucinating?
You often can't tell from the output alone — hallucinations look like correct responses. The practical answer: assume any specific factual claim could be hallucinated and verify independently when the stakes are consequential. Models may express uncertainty ("I'm not sure about this"), but expressed confidence doesn't correlate reliably with accuracy.
Key Takeaways
- AI hallucination is when an LLM generates confident, fluent, factually incorrect or fabricated text.
- Root cause: LLMs predict probable text based on training patterns, not verified truth — they have no internal signal for what they don't know.
- Common types: factual errors, citation fabrication, confabulation beyond provided context, entity invention, consistency errors.
- Citation fabrication is particularly dangerous — AI-generated references to papers, cases, and studies may not exist.
- Mitigation: RAG grounding, uncertainty prompting, verification pipelines, self-consistency checking.
- Working assumption: treat LLM outputs as drafts requiring verification, not authoritative sources.
Conclusion
AI hallucination is the central limitation of using large language models for anything where factual accuracy matters. It's not a bug that will be fixed in the next model update — it's an inherent consequence of how these systems generate text. The practical response is a verification habit: use LLMs for their strengths (drafting, summarizing, explaining, structuring, generating) while independently verifying specific facts, citations, and claims before using them in any consequential context. The knowledge workers who use LLMs effectively are those who understand this constraint and build verification into their workflow.
Try WebSnips free — build your own verified knowledge base of saved, sourced, annotated research that you can trust: the antidote to AI-generated hallucinations when you need reliable information.