Versatile AI Risk Assessment

AI threat catalogue Reliability and Responsible Use Production

Factual Inconsistencies (Hallucinations)

AI language models sometimes produce content that sounds convincing but is simply made up (hallucinations), including citations, figures, sources, or events. These outputs appear just as confident as correct answers.

As of: October 2026 · Catalogue version 2026.10.24.2 · 6 mitigations · 11 verified sources

Description

Language models compute, word by word, the statistically most likely continuation of a text. They do not check whether a statement is true and have no concept of the difference between knowing and inventing. Where the model lacks information, it fills the gap with plausible-sounding but fabricated content, including invented scientific references, court rulings, statistics, or software libraries. This threat needs no attacker: it is a property of the technology and can occur in any deployment, especially for questions at the edge of its trained knowledge. It becomes risky wherever outputs flow unchecked into advice, legal matters, medicine, or journalistic content.

Possible impact

When employees or customers act on fabricated content, the result is poor decisions, rework, and liability exposure; in a publicly documented legal dispute, a company has already been held to its chatbot's incorrect statement. Invented statements about individuals also touch the GDPR principle of accuracy. Publicly known incidents damage trust in the company and its AI services.

Example

An airline's customer chatbot explained a refund policy to a traveller that had never existed. The company lost the ensuing legal dispute and had to honour the invented commitment; the case is publicly documented.

Recommended mitigations (6)

Compare effects and implementation levels. Expand each entry for the additional classification and rationale.

  • Retrieval-augmented generation (RAG)

    Technical
    Effect
    Preventive
    Implementation level
    Data, Application, API & agents
    Classification and rationale

    Primarily technical: A model, training, or data-processing method directly changes system behavior or robustness.

  • Source attribution in outputs

    Technical
    Effect
    Preventive, Detective
    Implementation level
    Application, API & agents, Use & operations
    Classification and rationale

    Primarily technical: Cryptographic or machine-verifiable properties protect confidentiality, integrity, or provenance.

  • Confidence scoring

    Technical
    Effect
    Preventive, Detective
    Implementation level
    Application, API & agents, Use & operations
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

  • Fact-checking post-processing

    Technical
    Effect
    Detective
    Implementation level
    Application, API & agents, Use & operations
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

  • Clear user disclaimers

    Technical
    Effect
    Preventive
    Implementation level
    Application, API & agents, Use & operations
    Classification and rationale

    Complementary control typePeople & competence

    Primarily technical: The application makes uncertainty, system boundaries, or safe next steps visible and supports informed decisions; complemented by human expertise and judgment.

  • Avoid overconfidence in model training

    Technical
    Effect
    Preventive
    Implementation level
    Model & training
    Classification and rationale

    Primarily technical: A model, training, or data-processing method directly changes system behavior or robustness.

Framework mappings

Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.

OWASP LLM Top 10 LLM04:2026 · LLM07:2026NIST AI RMF Section 2.2 · Section 2.8EU AI Act Article 13(1), 13(3)(b)(ii), (iv), (v) · Article 9(1), 9(2)(a), 9(2)(d)BSI R15 · R4BIML BIML-LLM inference:3 · BIML-LLM LLMtop10:9 · BIML-LLM raw:10

Verified references (11)

Every reference states the framework, the exact location and the publishing organisation.

EU AI Act obligations related to this threat

The obligations from the obligations compass whose articles the catalogue maps to this threat. Each leads to its entry with core, action step and evidence; whether it applies to you depends on role and risk class.

  • Establish and operate a risk management system Art. 9 · applies from Annex III: 2 December 2027 · Annex I: 2 August 2028

    Statutory roles: Provider, Product manufacturer. Classes: High-risk under Annex III, High-risk as a product or safety component.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Provide instructions for use to deployers Art. 13 · applies from Annex III: 2 December 2027 · Annex I: 2 August 2028

    Statutory roles: Provider, Product manufacturer. Classes: High-risk under Annex III, High-risk as a product or safety component.

    This reference is a review prompt. A threat match alone does not establish applicability.

All 62 obligations by role, risk class and deadline in the obligations compass

Terms on this page

Glossary terms that occur in this entry. Every link leads to the full explanation.

More entries from the topic group Reliability and Responsible Use.

Overreliance People tend to accept convincing-sounding AI outputs without checking them (automation bias). Model Drift & Degradation The quality of an AI model gradually declines in operation because the world keeps changing while the model stays frozen on old knowledge … Shadow AI (Unsanctioned AI Service Use) Employees or business units use AI services without company approval or oversight (shadow AI).

Matching use cases

This list shows where to review this threat, rather than how high the risk is.

Particularly important for:

  • Generative AI assistant for staffA draft cites a nonexistent provision, judgment or figure. If someone uses the text, the error enters correspondence to customers or citizens.
  • AI knowledge assistant (enterprise search)If retrieval finds no suitable passage, the language model still produces an answer. An invented security rule or incorrect work step then looks like a manual quotation.
  • Speech recognition and transcriptionDuring a pause or unclear speech, the model inserts a sentence never spoken. In a finding or decision, it is later treated as the person's statement.
  • AI meeting minutes and conversation documentationThe language model attributes a statement to a client that they never made or adds a diagnosis. Once in the case file, the sentence is treated as established in later decisions.
  • Machine translationThe model omits a negative or mistranslates a technical term while still sounding fluent. The meaning of a decision or instruction changes unnoticed.
  • Text analytics for feedback and consultation responsesThe model summarises a theme barely present or presents a minority view as the majority. Readers relying only on the summary draw incorrect policy or product conclusions.

Typical for:

For 10 further use cases, relevance depends on the system design, for example whether it generates its own text. Show these use cases

Frequently asked questions

What is Factual Inconsistencies (Hallucinations)?
AI language models sometimes produce content that sounds convincing but is simply made up (hallucinations), including citations, figures, sources, or events. These outputs appear just as confident as correct answers.
Which mitigations help against Factual Inconsistencies (Hallucinations)?
The catalogue lists 6 mitigations, each with control type, effect and implementation level. First among them are Retrieval-augmented generation (RAG), Source attribution in outputs and Confidence scoring.
Which EU AI Act obligations does Factual Inconsistencies (Hallucinations) touch?
The catalogue maps this threat to Art. 9 and Art. 13 of the EU AI Act. In the obligations compass this leads to: Establish and operate a risk management system and Provide instructions for use to deployers. Whether an obligation applies depends on role, risk class and deadline.

Assess this threat in your own system

The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.

Start the live demo See the free modules

Cite this entry

For reports, policies or internal documents; the link leads directly to this entry.

“Factual Inconsistencies (Hallucinations)”. Versatile AI Risk Assessment, AI threat catalogue, as of October 2026.
https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/hallucinations/

← Back to the full catalogue