What it means
Language models generate text by predicting what comes next. Most of the time that produces accurate answers. Sometimes it produces fluent, confident text that is simply wrong: a policy that does not exist, a wrong number or a citation to a page nobody wrote.
Hallucinations are more likely when a model is asked about things it was not given, such as your internal policies, or when it is pushed to answer when it should say it does not know.
Why it matters for a business
A wrong answer delivered confidently can mislead a customer or a colleague. The fix is design, not hope: ground answers in your own documents, show sources, allow "I do not know" and test with real questions before launch.
A business example
Things to watch
-
Ground answers in your own documents with retrieval.
-
Ask the model to cite sources and check them.
-
Let the assistant say it does not know.
-
Test with the hardest real questions before launch.