Versatile AI Risk Assessment

AI threat catalogue Malicious Use for Attacks, Fraud and Disinformation Production

Disinformation

AI makes it possible to mass-produce false or misleading content of convincing quality: fabricated news articles, manipulated images and videos, and coordinated sham campaigns on social media.

As of: October 2026 · Catalogue version 2026.10.24.2 · 5 mitigations · 8 verified sources

Description

Generative AI produces large volumes of credible-sounding false information in a short time: fabricated news articles, propaganda material, fake reviews, manipulated imagery and deepfakes (deceptively real AI forgeries of images, audio or video of real people). Added to this is astroturfing: coordinated campaigns in which many seemingly independent accounts create the impression of broad public opinion. Behind such campaigns are state-directed influence operations as well as commercially motivated actors; the content spreads primarily through social media and review platforms. For companies, both sides matter: they can become the target of such campaigns themselves, and at the same time trust in genuine content declines overall (erosion of trust).

Possible impact

Targeted false reports or deepfakes about the company or its executives can quickly damage reputation, customer relationships and share prices. Fake reviews distort competition. Corrections take time and rarely reach everyone who saw the original report; where the company itself publishes AI-generated content, labelling and disclosure obligations may apply, especially for deepfakes. Such campaigns also endanger democratic discourse and public trust.

Example

An AI-generated video in which the CEO appears to issue a profit warning spreads on social media. By the time it is corrected, customers, media and investors have already reacted to the entirely fabricated statement.

Recommended mitigations (5)

Compare effects and implementation levels. Expand each entry for the additional classification and rationale.

  • Provenance and watermarking of generated content

    Technical
    Effect
    Preventive, Detective
    Implementation level
    Application, API & agents, Use & operations
    Classification and rationale

    Complementary control typeOrganizational & process-based

    Primarily technical: Cryptographic or machine-verifiable properties protect confidentiality, integrity, or provenance; complemented by binding workflows.

  • Fact-checking integration

    Technical
    Effect
    Detective
    Implementation level
    Application, API & agents, Use & operations
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

  • Detection of coordinated inauthentic behavior

    Technical
    Effect
    Detective
    Implementation level
    Application, API & agents, Use & operations
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

  • Disclosure requirements

    Governance & compliance
    Effect
    Preventive
    Implementation level
    Organization, Use & operations
    Classification and rationale

    Complementary control typeOrganizational & process-based

    Primarily a governance and compliance control: Binding rules, control objectives, or oversight define permitted use and accountability; complemented by binding workflows.

  • Platform abuse detection

    Technical
    Effect
    Detective
    Implementation level
    Application, API & agents, Use & operations
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

Framework mappings

Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.

OWASP LLM Top 10 LLM07:2026NIST AI RMF Section 2.8MITRE ATLAS AML.T0048.002EU AI Act Article 5(1)(a) · Article 50(2), 50(4) · Article 55(1)(b)BSI R10 · R5

Verified references (8)

Every reference states the framework, the exact location and the publishing organisation.

EU AI Act obligations related to this threat

The obligations from the obligations compass whose articles the catalogue maps to this threat. Each leads to its entry with core, action step and evidence; whether it applies to you depends on role and risk class.

  • Refrain from the prohibited practices Art. 5 Abs. 1 · applies from 2 February 2025

    Statutory roles: Provider, Deployer, Importer, Distributor, Authorised representative, Product manufacturer, Provider of a general-purpose AI model. Classes: Prohibited practice.

    These prohibitions have applied since 2 February 2025 and are unaffected by the deferral of the high-risk obligations. Infringements carry the highest fine bracket of the regulation.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • New prohibition: intimate depictions without consent Art. 5 Abs. 1 UAbs. 1 Buchst. ba, Art. 5 Abs. 1a, 1b · applies from 2 December 2026 · amended by Reg. 2026/1744

    Statutory roles: Provider, Deployer, Importer, Distributor, Authorised representative, Product manufacturer, Provider of a general-purpose AI model. Classes: Prohibited practice, System subject to transparency obligations.

    This prohibition is new and was inserted by Regulation (EU) 2026/1744. It applies from 2 December 2026. For deployers, the prohibition applies only where the system is used for that very purpose.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • New prohibition: child sexual abuse material Art. 5 Abs. 1 UAbs. 1 Buchst. bb · applies from 2 December 2026 · amended by Reg. 2026/1744

    Statutory roles: Provider, Deployer, Importer, Distributor, Authorised representative, Product manufacturer, Provider of a general-purpose AI model. Classes: Prohibited practice.

    Also newly inserted by Regulation (EU) 2026/1744, applying from 2 December 2026.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Mark synthetic content in machine-readable form Art. 50 Abs. 2 · applies from 2 August 2026

    Statutory roles: Provider, Product manufacturer, Provider of a general-purpose AI model. Classes: System subject to transparency obligations, High-risk under Annex III, High-risk as a product or safety component, General-purpose AI model, Model with systemic risk.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Label deepfakes and published AI texts Art. 50 Abs. 4 · applies from 2 August 2026

    Statutory roles: Deployer. Classes: System subject to transparency obligations, High-risk under Annex III, High-risk as a product or safety component, Minimal risk.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Evaluate and mitigate systemic risks Art. 55 · applies from 2 August 2025

    Statutory roles: Provider of a general-purpose AI model. Classes: Model with systemic risk.

    This reference is a review prompt. A threat match alone does not establish applicability.

All 62 obligations by role, risk class and deadline in the obligations compass

More entries from the topic group Malicious Use for Attacks, Fraud and Disinformation.

Social Engineering Attackers use AI to produce deceptively authentic, personally tailored scam messages, calls and pretext stories at scale. Fraud Criminals use AI to commit fraud at greater scale and with greater credibility: through fake reviews, invented identities, forged … Malicious Software Attackers use AI models to generate or improve working malware and attack tooling.

Matching use cases

This list shows where to review this threat, rather than how high the risk is.

Particularly important for:

  • Image, video and voice generationAn advertising video shows a realistic scene involving a well-known person that never happened. Without AI disclosure, viewers believe it is real and share it further.

Typical for:

For 6 further use cases, relevance depends on the system design, for example whether it generates new content from open-ended requests or generates its own text. Show these use cases

Frequently asked questions

What is Disinformation?
AI makes it possible to mass-produce false or misleading content of convincing quality: fabricated news articles, manipulated images and videos, and coordinated sham campaigns on social media.
Which mitigations help against Disinformation?
The catalogue lists 5 mitigations, each with control type, effect and implementation level. First among them are Provenance and watermarking of generated content, Fact-checking integration and Detection of coordinated inauthentic behavior.
Which EU AI Act obligations does Disinformation touch?
The catalogue maps this threat to Art. 5 Abs. 1, Art. 5 Abs. 1 UAbs. 1 Buchst. ba, Art. 5 Abs. 1 UAbs. 1 Buchst. bb, Art. 50 Abs. 2, Art. 50 Abs. 4 and Art. 55 of the EU AI Act. In the obligations compass this leads to: Refrain from the prohibited practices, New prohibition: intimate depictions without consent, New prohibition: child sexual abuse material, Mark synthetic content in machine-readable form, Label deepfakes and published AI texts and Evaluate and mitigate systemic risks. Whether an obligation applies depends on role, risk class and deadline.

Assess this threat in your own system

The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.

Start the live demo See the free modules

Cite this entry

For reports, policies or internal documents; the link leads directly to this entry.

“Disinformation”. Versatile AI Risk Assessment, AI threat catalogue, as of October 2026.
https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/disinformation/

← Back to the full catalogue