What You Will Learn
INTERMEDIATE
- Why hallucination is a feature, not a bug — built into how LLMs work
- The top 5 causes of hallucination in production systems
- 7 battle-tested techniques to minimize hallucinations in your apps
The Uncomfortable Truth
Here's the truth that every LLM engineer eventually accepts: LLMs will always hallucinate. Not because they're broken, but because of what they fundamentally are — next-token predictors. They generate text that sounds right, not text that is right. There's no fact-checker inside the model. No database it consults. No notion of "truth" — only patterns learned from training data.
This isn't a design flaw to be patched; it's the core mechanism. The same generative power that lets an LLM write beautiful poetry lets it confidently invent facts. You can't have one without the other. So instead of asking "how do we eliminate hallucination?" the right question is "how do we minimize it for our use case?"
The Top 5 Causes of Hallucination
1. Training Data Contained Wrong Information
If the model read a Wikipedia article that said Marie Curie won the Nobel Prize in 1911 (correct) and a blog that said she won it in 1912 (wrong), it has both patterns in its weights. When asked, it might output either date — or worse, blend them ("around 1911-1912"). The model has no way to know which source is more authoritative.
2. The Model Doesn't Know What It Doesn't Know
LLMs have no calibrated sense of uncertainty. A model will produce a confident-sounding answer whether it's 99% sure or 5% sure. There's no internal "I don't know" flag that fires reliably. Asking the model to express uncertainty helps a little, but it still hallucinates confidently about half the time.
3. Prompt Induces Fabrication
If you ask "What did Albert Einstein say about quantum entanglement in his 1935 paper?", the model might quote something plausible-sounding — even if Einstein never said it. The question presupposes a quote exists, so the model generates one. This is called presupposition-induced hallucination, and it's why "what is X?" works better than "tell me about X".
4. Long-Context Forgetting
Even with 1M-token contexts, attention is O(n²), and models effectively "lose" information in the middle of long inputs (the "lost in the middle" phenomenon). Put your most important context at the start and end of the prompt, not buried in paragraph 47.
5. Sampling Randomness
High temperature means the model picks from less-likely tokens, which is great for creativity but terrible for accuracy. For factual tasks, use temperature=0 (or low, like 0.1-0.3). Most production hallucinations in early demos were just temperature set too high.
| Cause | Example | Fix |
|---|---|---|
| Wrong training data | Says Curie won Nobel in 1912 | Use RAG with verified sources |
| No uncertainty calibration | Confidently wrong answers | Add 'If unsure, say I don't know' to system prompt |
| Presupposition in prompt | Fabricates Einstein quotes | Ask open questions: 'Did Einstein say X about Y?' |
| Lost in the middle | Forgets facts in long context | Put key info at start AND end of prompt |
| High temperature | Creative but inaccurate text | temperature=0 for facts, 0.7+ for creativity |
7 Techniques to Minimize Hallucination
1. Use RAG (Retrieval-Augmented Generation)
The single most effective technique. Instead of relying on the model's internal knowledge, retrieve relevant documents from a verified source, put them in the prompt, and ask the model to answer only from those documents. This converts the model from "guessing" to "reading comprehension". We'll build a full RAG system in Lesson 34.
2. Ask for Citations
Instruct the model to cite sources for each claim. If it can't cite a source, it should say so. This forces the model to either ground claims or admit uncertainty:
system_prompt = """
Answer the user's question. For every factual claim, include a citation in [square brackets].
If you cannot verify a claim with high confidence, write: [UNVERIFIED].
Prefer saying 'I don't know' over guessing.
"""
3. Two-Step Verification
Have the model generate an answer, then have it (or another model) verify the answer against the source documents. This catches many hallucinations because the verifier can see things the generator missed.
4. Lower Temperature for Factual Tasks
Use temperature=0 (or 0.1) for factual Q&A, summaries, classification. Use higher temperature (0.7+) only for creative tasks like writing, brainstorming, dialogue. Many hallucinations in early demos were just temperature set wrong.
5. Use Function Calling for Facts
For things the model can't know (current weather, stock prices, math), give it a tool. Let it call get_weather() or calculate() instead of guessing. The model decides when to use the tool; the tool itself is reliable.
6. Constrain Output with Schemas
Force structured output (JSON with defined schema) instead of free text. Hallucination is much less likely when the model must fit its answer into a strict structure. Use OpenAI's response_format: json_schema or libraries like Instructor.
7. Use a Smaller, Specialized Model
For narrow domains (legal, medical, internal docs), a smaller model fine-tuned on your domain often hallucinates less than a general-purpose model. General models have broader but shallower knowledge; fine-tuned models have deeper but narrower knowledge.
Practical Exercise
🎯 Key Takeaways
- Hallucination is built into how LLMs work — they generate plausible text, not verified facts.
- 5 causes: bad training data, no uncertainty calibration, prompt presuppositions, lost-in-middle, high temperature.
- Best fix: RAG — retrieve verified docs, ask the model to answer only from them.
- Other fixes: citations, two-step verification, low temperature, function calling, structured output, specialized models.
- If your app needs zero hallucination, LLMs are the wrong tool — use a database.
Comments
Comments
Post a Comment