Language models generate likely text, not verified facts. When they lack the right information, or the question invites invention, they can produce confident but incorrect statements, citations that do not exist, or numbers that were never in the source. This is commonly called hallucination.
Practical mitigations include grounding answers in retrieved sources, instructing the model to say when it does not know, checking outputs against rules or source data, evaluating on real examples, and keeping a person in the loop for consequential output. No current technique eliminates the risk entirely.