A hallucination is when the model produces incorrect information just as confidently as correct information: a nonexistent library method, a made-up link, a fact that never happened. For a product engineer this isn't a rare bug — it's a basic property you have to work with every day.

Hallucinations are easiest to understand if you remember how the model works: it predicts a plausible continuation rather than checking for truth. Plausible and true usually coincide — but not always.

Why it happens

The model always outputs something, because its job is to continue text. If it doesn't have the exact answer in its "memory", it won't say "I don't know" by default — it'll substitute the most plausible-looking variant. A made-up method parseDateSafe() looks exactly like a method of such a library should, so the model confidently "recalls" it, even though it doesn't exist.

The key trap: confidence of tone is unrelated to correctness. The model sounds equally confident whether it's reproducing a real fact or constructing a nonexistent one. Tone is not a signal of truth.

Where hallucinations are most likely

The risk isn't uniform. It rises sharply where there was little data, or where there couldn't be any:

  • Rare, niche APIs — obscure libraries, exotic flags. The model knows popular code; rare code it fabricates.
  • Fresh versions and events — anything after the model's training cutoff. It doesn't know what changed and answers from the old state.
  • Precise facts — specific numbers, dates, quotes, links, names. Here it's especially easy to get a plausible fabrication.
  • Your private context — internal code, documents, team agreements. The model hasn't seen them; if you don't put them in context, it'll "fill them in".
  • Overly general prompts — the less to anchor to in the question, the more the model constructs itself.

How to reduce the risk

You can't eliminate hallucinations entirely — you can sharply reduce their likelihood and cost:

  • Ground in data. Don't ask "from memory" — give the relevant documentation, code, or facts right in the prompt. Then the model relies on them, not on probabilities. The same is done by external sources via tool calling (search, database access).
  • Ask for the verifiable. Require source links, specific places in the code, justification. A fabrication is easier to catch when it can be checked.
  • Verify the output. Compile it, run the tests, check against real documentation — verifying AI output should be part of the process, not an option. This especially applies to reviewing AI code.
  • Allow "I don't know". Explicitly tell the model that admitting ignorance is better than making something up. This reduces confident errors.
  • Don't trust facts without checking. Numbers, links, API names, quotes — always verify against the primary source, however convincing it sounds.

What this means in practice

Hallucinations aren't a reason to abandon AI — they're a reason to set up the work so a fabrication never reaches the product. A product engineer treats any model output as a draft from a very well-read but occasionally fantasizing assistant: useful, fast, but verified before it goes into action. The higher the cost of an error (facts for users, money, security), the stricter the check.

What's next

Hallucinations grow when the model works without the right grounding. Learn how that grounding is managed: context (what to give the model), tool calling (how to give access to real data), and verifying AI output.