Hallucination
An AI system producing output that sounds fluent and confident but is factually wrong — inventing a detail rather than reporting one that's actually true.
NIST's Generative AI Profile names this failure mode — officially termed confabulation — as one of twelve formally tracked generative-AI risks. It's a structural property of how large language models generate text (predicting a plausible next word, not looking a fact up), not a rare glitch specific to a poorly built system, and confidence and correctness are entirely unrelated in these systems.
A caller asks whether a restaurant makes a dish they remember from somewhere else, one that genuinely isn't on this restaurant's menu. A system with nothing anchoring its answers to the real menu has every structural incentive to produce a plausible-sounding “yes, we do” — plausible language is exactly what these systems generate.
The most consequential mistake is treating hallucination as a rare malfunction rather than a property present to some degree in every system built this way. The fix that actually matters isn't a smarter model — it's architectural: keeping the part that decides real, checkable facts separate from the part generating conversational language.
The conversational side of Dohos can interpret what a caller says, but only a separate, deterministic backend decides what something actually costs or whether an order can go through — the boundary covered on Development practices.
BACK TO THE FULL GLOSSARY · OR THE WORKED GUIDES
Open the line.
Tell us about your restaurant. We load your menu, you place a call, and you hear it answered yourself.