What Is Hallucination in AI? – Definition and How It Works
Learn what AI hallucination is, why next-token prediction produces confident fabrication, the types that occur, and what actually reduces it.
Hallucination is a model producing content that is fluent, plausible, and false. It is not a malfunction; it is the expected behaviour of a system trained to produce likely continuations rather than verified statements.
1. Why It Happens
A language model predicts the next token. Nothing in that objective distinguishes "true" from "reads like the truth".
- No knowledge boundary. The model has no representation of what it does not know.
- No lookup step. Without tools it cannot check anything.
- Fluency is optimised, accuracy is not. A well-formed wrong answer scores well on the training objective.
The most confident-sounding output is therefore not correlated with correctness in the way readers assume.
2. The Types
| Type | Example |
|---|---|
| Factual | A wrong date, figure, or attribution |
| Citation | A plausible-looking source that does not exist |
| Code | A function or parameter that was never in the library |
| Extrinsic | Details added that were not in the provided context |
| Instruction | Claiming to have performed an action it cannot perform |
Citation hallucination is the most damaging in practice, because a fabricated reference looks exactly like a real one.
3. When It Is Most Likely
- Questions about events after the training cutoff.
- Niche topics with thin training coverage.
- Requests for specific figures, dates, or identifiers.
- Prompts that presuppose something false, which the model tends to accept.
- Long generations, where early inventions are elaborated on later.
4. What Reduces It
Grounding. Retrieve real source text and instruct the model to answer only from it, and to say when the context is insufficient. This is the single largest reduction available.
Tools. Let the model look things up, calculate, and query rather than recall.
Structured output. A schema constrains the shape and makes validation mechanical.
Refusal permission. Explicitly allowing "I don't know" makes it a much more likely output.
Verification. Check claims against a source programmatically where they matter.
5. What Does Not Reliably Reduce It
- Asking for confidence. Stated confidence is generated text like any other.
- "Do not hallucinate." An instruction cannot create a knowledge boundary that does not exist.
- Larger models. They hallucinate less on common topics and still fabricate on thin ones.
- Lower temperature. It reduces variety, not error.
6. Detecting It
- Verify every citation resolves and says what was claimed.
- Cross-check figures against the retrieved context.
- Re-run the same prompt: inconsistent answers between runs indicate the model is generating rather than recalling.
- Validate structured output against a schema and reject mismatches.
7. Designing Around It
Treat model output as a draft requiring verification wherever it is load-bearing. Systems that work in production put the facts in the context window and use the model for transformation and phrasing, rather than asking it to supply the facts.
That framing also explains why retrieval, tools, and validation dominate serious deployments.
8. Summary
Hallucination is confident, fluent invention, and it follows directly from an objective that rewards plausibility rather than truth.
Ground answers in retrieved text, give the model tools instead of asking it to recall, permit refusal explicitly, and verify anything that matters — instructions alone will not fix a limitation that is architectural.