Limitations & Hallucinations
Because an LLM predicts plausible next tokens rather than checking truth, it can state false things with total confidence, is frozen at its training cutoff, and has no live access to your data or the internet — so knowing its limits and how to work around them is a core skill.
By now you know the machine underneath: an LLM is a next-token predictor. Given some text, it produces a probability distribution over what comes next, samples a token, and repeats. That single mechanism is astonishingly powerful — and it's also the source of every serious limitation in this lesson.
The honest version of the story is this: *the model optimizes for plausible, not for true. Most of the time plausible and true line up, which is why LLMs feel so capable. But when they diverge, the model will happily hand you something that sounds* right and is completely wrong — with the exact same confident tone it uses when it's correct. Knowing where and why that happens is what separates a careful user from a burned one.
#Hallucination: confident, fluent, and wrong
A hallucination is when a model states something false as if it were fact — a made-up citation, a nonexistent function, a wrong date, a plausible but fabricated quote. It's not lying (that would require knowing the truth and hiding it) and it's not a bug in the usual sense. It's the mechanism working exactly as designed.
Remember what training rewarded: producing text that looks like the text humans write. Nothing in that objective is a fact-checker. There is no internal database the model consults, no little green 'verified' light. When you ask for the title of a 2019 paper by a specific author, the model doesn't look it up — it generates the sequence of tokens that most plausibly follows your question. If a real title is well-represented in its training data, that plausible sequence is the real title. If it isn't, the model still produces a plausible-sounding title, because a plausible title is exactly what the pattern demands.
The improv actor with no script
Picture a brilliant improv actor playing 'world expert.' Ask a question and they will always stay in character and answer smoothly — that's the whole game. If they happen to know the fact, great, the answer is right. If they don't, they don't break character to say 'I have no idea'; they invent something that fits the scene perfectly. The fluency is identical either way. An LLM is that actor: it never steps off stage to check whether the line it's about to deliver is actually true.
Confidence is not calibration
The most dangerous part of hallucination is that the tone doesn't change. A wrong answer arrives with the same polished, assured phrasing as a right one. The model has no reliable internal signal of 'I'm unsure here' that surfaces as hedging. So you cannot use how confident it sounds as a proxy for how likely it is to be correct. Treat fluency and certainty as style, not evidence.
You ask an LLM for a citation and it returns a real-sounding author, title, journal, and year — but the paper doesn't exist. Why did this happen?
#Knowledge cutoff: frozen in time
A model's knowledge comes entirely from its training data, which was collected up to a certain point — its knowledge cutoff. After training, the weights are frozen. The model does not keep learning from your conversations, and it does not know anything that happened after that date.
So if you ask about a product released last week, an election held yesterday, or a library version shipped this morning, the model is answering from a snapshot of the past. Worse, it often won't say it's out of date — it'll answer confidently from stale knowledge, or hallucinate to fill the gap. 'What's the latest version of X?' is one of the most reliable ways to get a wrong, outdated, or invented answer.
Training data collected ──────────────►│ (knowledge cutoff)
│
Everything here the model "knows" │ Everything here is
(frozen into its weights) │ INVISIBLE to the model:
│ - news after the cutoff
│ - your private files
│ - today's stock price
│ - that API you shipped today
You ask a question ─────────────────────────────► ● (now)#No live access — to anything — by default
This one surprises people. A bare LLM cannot browse the web, read your files, run code, query your database, or check today's weather. On its own it is a closed box: text in, text out. It has no senses and no hands.
So when a model 'answers a question about your company's Q3 numbers,' one of two things is true. Either (a) those numbers were pasted into the prompt and it's reading them from the context window, or (b) it's making them up. There is no third option where the model 'went and looked.' Any real capability to fetch fresh or private information has to be given to the model explicitly — which is exactly what tools do (more on that in a moment).
"It answered, so it must have checked" — no
Getting a specific, detailed answer feels like proof the model consulted a source. It isn't. Unless a tool or retrieval step is wired in (and you can usually tell — there'll be a search step, a browsing indicator, or a document you provided), a plain model is generating from its frozen weights and whatever is in the prompt. Specificity is not sourcing.
#The rest of the limitation list
Beyond hallucination, staleness, and no live access, a few more weaknesses fall out of the same 'predict plausible tokens' design:
- Bias. The model learned from human-written text, so it absorbs the patterns, stereotypes, and skews present in that data. It can reproduce them, sometimes subtly.
- Prompt sensitivity. Reword a question and you can get a meaningfully different answer. Because output is a probabilistic function of the input tokens, small phrasing changes ('explain simply' vs 'explain rigorously', or even word order) can shift the result.
- Math and precise recall. The model isn't a calculator; it predicts what a plausible answer looks like. Multi-step arithmetic, exact figures, long ID numbers, and verbatim quotes are all shaky, because 'plausible-looking' and 'exactly correct' aren't the same target.
- No true memory across chats. As you saw with context windows, the model is stateless. It doesn't remember you between conversations unless something outside the model stores and re-supplies that history.
Why 2 + 2 can go wrong
It feels absurd that a system that writes flawless essays can flub arithmetic. But it's the same mechanism: the model isn't computing 47 × 89, it's predicting the tokens that tend to follow that expression. For small, common sums the pattern is everywhere in training data and it's usually right. For larger or unusual calculations the pattern is sparse, so it produces a plausible-looking number that may be off. The fix isn't a smarter guess — it's handing the model an actual calculator (a tool).
#Mitigations: how to work around the limits
None of this makes LLMs unusable — it makes them tools you drive carefully. The standard mitigations all share one idea: stop asking the model to be a source of truth, and start feeding it truth.
- Grounding / retrieval (RAG). Fetch the relevant, up-to-date documents and put them in the prompt, then ask the model to answer from those. Now the facts come from your source, not the model's memory.
- Citations. Have the model point to where each claim came from, so a human (or another check) can verify it instead of trusting fluency.
- Verification. For anything that matters, check the output — run the code, confirm the number, click the link. Treat the model's answer as a draft to be validated, not a verdict.
- Give the model tools. Let it call a calculator for math, a search engine for fresh facts, a database for private data, a file reader for your documents. A model with the right tool stops guessing and starts fetching.
Which of these actually attacks the *root cause* of both hallucination and knowledge-cutoff staleness?
Next up: connecting the model to real tools and data (MCP)
Notice the common thread in every mitigation: the model gets better the moment you connect it to something outside itself — a document store, a calculator, a live database, the web. The obvious next question is how you plug those in cleanly and safely. That's the entire subject of the next course, the Model Context Protocol (MCP) — a standard way to give an LLM real tools and real data so it can look things up instead of guessing. Everything you just learned about limits is the reason MCP exists.
Key takeaways
- LLMs optimize for plausible tokens, not truth — so they can hallucinate false facts, citations, and code with complete, unwavering confidence.
- A model's knowledge is frozen at its training cutoff; it doesn't know recent events and won't reliably tell you it's out of date.
- By default a bare LLM has no live access to the web, your files, tools, or private data — a detailed answer is not proof it 'looked anything up.'
- Related weaknesses — bias, prompt sensitivity, shaky math and exact recall — all trace back to the same predict-plausible-text mechanism.
- The fix is to feed the model truth rather than trust its memory: grounding/retrieval (RAG), citations, human verification, and giving it real tools.
A teammate wrote a checklist explaining why LLMs hallucinate. Exactly one line is a real misconception. Which one is WRONG?
# Why do LLMs hallucinate?
1. The model predicts plausible next tokens, not verified facts.
2. There is no built-in fact-checker comparing output to a source of truth.
3. When it hallucinates, the model KNOWS it's unsure and clearly warns you.
4. Well-formed patterns (like citations) are easy to fabricate convincingly.You send this to a PLAIN LLM — no tools, no web access, no retrieval. Its training cutoff was in 2024. What's the honest description of the answer you'll get?
prompt = "What is the current top news story right now,
today, and who is the source?"
# model: bare LLM, knowledge cutoff = 2024,
# no browsing / no tools enabledFill in the blanks to complete the summary of LLM limits and their fixes.
A model's knowledge is frozen at its training , so it's stale on recent events. Because it predicts plausible text with no fact-checker, it can — state false things confidently. The root-cause fix is to feed it truth: the answer in retrieved documents, or give the model real .
Order the steps of a grounded (retrieval-augmented) answer that avoids hallucinating an out-of-date fact — the pattern that fixes the bare model's limits.
The answer includes citations so a human can verify each claim
The model generates an answer grounded in the supplied documents
Retrieve the relevant, up-to-date documents from a trusted source
User asks a question that needs current or private information
Insert those documents into the model's prompt (the context window)
A colleague builds an internal assistant on top of a plain LLM (no tools, no retrieval) and asks it three questions. For each, decide whether the answer is trustworthy as-is, and name the specific limitation at play if not:
- "Summarize the key argument of this contract" — with the full contract text pasted into the prompt.
- "What is the current version number of our internal billing library, released two days ago?"
- "Multiply these two 6-digit invoice totals and give me the exact result."
Then: pick the one question whose answer you'd trust most, and explain in one sentence why the mechanism makes it safer than the others.
Try it yourself — a starting point to build on:
# Write your solution here