AI threat catalogueReliability and Responsible UseProduction
Factual Inconsistencies (Hallucinations)
AI language models sometimes produce content that sounds convincing but is simply made up (hallucinations), including citations, figures, sources, or events. These outputs appear just as confident as correct answers.
Description
Language models compute, word by word, the statistically most likely continuation of a text. They do not check whether a statement is true and have no concept of the difference between knowing and inventing. Where the model lacks information, it fills the gap with plausible-sounding but fabricated content, including invented scientific references, court rulings, statistics, or software libraries. This threat needs no attacker: it is a property of the technology and can occur in any deployment, especially for questions at the edge of its trained knowledge. It becomes risky wherever outputs flow unchecked into advice, legal matters, medicine, or journalistic content.
Possible impact
When employees or customers act on fabricated content, the result is poor decisions, rework, and liability exposure; in a publicly documented legal dispute, a company has already been held to its chatbot's incorrect statement. Invented statements about individuals also touch the GDPR principle of accuracy. Publicly known incidents damage trust in the company and its AI services.
Example
An airline's customer chatbot explained a refund policy to a traveller that had never existed. The company lost the ensuing legal dispute and had to honour the invented commitment; the case is publicly documented.
Recommended mitigations (6)
Every mitigation states its control type, effect, implementation level and the reason for the classification.
Retrieval-augmented generation (RAG)Technical
- Effect
- Preventive
- Implementation level
- Data, Application, API & agents
- Reason for the classification
- “Retrieval-augmented generation (RAG)” is primarily technical: A model, training, or data-processing method directly changes system behavior or robustness.
Source attribution in outputsTechnical
- Effect
- Preventive, Detective
- Implementation level
- Application, API & agents, Use & operations
- Reason for the classification
- “Source attribution in outputs” is primarily technical: Cryptographic or machine-verifiable properties protect confidentiality, integrity, or provenance.
Confidence scoringTechnical
- Effect
- Preventive, Detective
- Implementation level
- Application, API & agents, Use & operations
- Reason for the classification
- “Confidence scoring” is primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.
Fact-checking post-processingTechnical
- Effect
- Detective
- Implementation level
- Application, API & agents, Use & operations
- Reason for the classification
- “Fact-checking post-processing” is primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.
Clear user disclaimersTechnical
- Effect
- Preventive
- Implementation level
- Application, API & agents, Use & operations
- Complementary control type
- People & competence
- Reason for the classification
- “Clear user disclaimers” is primarily technical: The application makes uncertainty, system boundaries, or safe next steps visible and supports informed decisions; complemented by human expertise and judgment.
Avoid overconfidence in model trainingTechnical
- Effect
- Preventive
- Implementation level
- Model & training
- Reason for the classification
- “Avoid overconfidence in model training” is primarily technical: A model, training, or data-processing method directly changes system behavior or robustness.
Framework mappings
Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.
Verified references (10)
Every reference states the framework, the exact location and the publishing organisation.
- OWASP LLM Top 10 LLM09:2025 MisinformationLLM09:2025 Misinformation, official category page OWASP FoundationOriginal
- NIST AI RMF Section 2.2 ConfabulationSection 2.2, p. 6 National Institute of Standards and Technology (NIST)Original
- NIST AI RMF Section 2.8 Information IntegritySection 2.8, pp. 9–10 National Institute of Standards and Technology (NIST)Original
- EU AI Act Article 13(1), 13(3)(b)(ii), (iv), (v) Transparency and provision of information to deployersArticle 13(1), 13(3)(b)(ii), (iv), (v) European Union (EUR-Lex)Original
- EU AI Act Article 9(1), 9(2)(a), 9(2)(d) Risk management systemArticle 9(1), 9(2)(a), 9(2)(d), read with Article 9(3) European Union (EUR-Lex)Original
- BSI R15 Platzierung von Malware (Text)Kap. 4, R15, p. 22 Bundesamt für Sicherheit in der Informationstechnik (BSI)Original
- BSI R4 Fehlende Ausgabequalität (Text, Bild, Video)Kap. 4, R4, p. 15 Bundesamt für Sicherheit in der Informationstechnik (BSI)Original
- BIML BIML-LLM inference:3 WrongnessPDF p. 18, [inference:3:wrongness] Berryville Institute of Machine Learning (BIML)Original
- BIML BIML-LLM LLMtop10:9 Model TrustworthinessPDF p. 13, [LLMtop10:9:model trustworthiness] Berryville Institute of Machine Learning (BIML)Original
- BIML BIML-LLM raw:10 Query DataPDF p. 15, [raw:10:query data] Berryville Institute of Machine Learning (BIML)Original
Terms on this page
Glossary terms that occur in this entry. Every link leads to the full explanation.
- RAG (retrieval augmented generation) The model pulls matching documents from your own knowledge base before answering.
Related threats
More entries from the topic group Reliability and Responsible Use.
Assess this threat in your own system
The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.
Cite this entry
For reports, policies or internal documents; the link leads directly to this entry.
“Factual Inconsistencies (Hallucinations)”. Versatile AI Risk Assessment, AI threat catalogue, as of July 2026. https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/hallucinations/