AI threat catalogueMalicious Use for Attacks, Fraud and DisinformationProduction
Disinformation
AI makes it possible to mass-produce false or misleading content of convincing quality: fabricated news articles, manipulated images and videos, and coordinated sham campaigns on social media.
Description
Generative AI produces large volumes of credible-sounding false information in a short time: fabricated news articles, propaganda material, fake reviews, manipulated imagery and deepfakes (deceptively real AI forgeries of images, audio or video of real people). Added to this is astroturfing: coordinated campaigns in which many seemingly independent accounts create the impression of broad public opinion. Behind such campaigns are state-directed influence operations as well as commercially motivated actors; the content spreads primarily through social media and review platforms. For companies, both sides matter: they can become the target of such campaigns themselves, and at the same time trust in genuine content declines overall (erosion of trust).
Possible impact
Targeted false reports or deepfakes about the company or its executives can quickly damage reputation, customer relationships and share prices. Fake reviews distort competition. Corrections take time and rarely reach everyone who saw the original report; where the company itself publishes AI-generated content, labelling and disclosure obligations may apply, especially for deepfakes. Such campaigns also endanger democratic discourse and public trust.
Example
An AI-generated video in which the CEO appears to issue a profit warning spreads on social media. By the time it is corrected, customers, media and investors have already reacted to the entirely fabricated statement.
Recommended mitigations (5)
Every mitigation states its control type, effect, implementation level and the reason for the classification.
Provenance and watermarking of generated contentTechnical
- Effect
- Preventive, Detective
- Implementation level
- Application, API & agents, Use & operations
- Complementary control type
- Organizational & process-based
- Reason for the classification
- “Provenance and watermarking of generated content” is primarily technical: Cryptographic or machine-verifiable properties protect confidentiality, integrity, or provenance; complemented by binding workflows.
Fact-checking integrationTechnical
- Effect
- Detective
- Implementation level
- Application, API & agents, Use & operations
- Reason for the classification
- “Fact-checking integration” is primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.
Detection of coordinated inauthentic behaviorTechnical
- Effect
- Detective
- Implementation level
- Application, API & agents, Use & operations
- Reason for the classification
- “Detection of coordinated inauthentic behavior” is primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.
Disclosure requirementsGovernance & compliance
- Effect
- Preventive
- Implementation level
- Organization, Use & operations
- Complementary control type
- Organizational & process-based
- Reason for the classification
- “Disclosure requirements” is primarily a governance and compliance control: Binding rules, control objectives, or oversight define permitted use and accountability; complemented by binding workflows.
Platform abuse detectionTechnical
- Effect
- Detective
- Implementation level
- Application, API & agents, Use & operations
- Reason for the classification
- “Platform abuse detection” is primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.
Framework mappings
Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.
Verified references (8)
Every reference states the framework, the exact location and the publishing organisation.
- OWASP LLM Top 10 LLM09:2025 MisinformationLLM09:2025 Misinformation, official category page OWASP FoundationOriginal
- NIST AI RMF Section 2.8 Information IntegritySection 2.8, pp. 9–10 National Institute of Standards and Technology (NIST)Original
- MITRE ATLAS AML.T0048.002 Societal HarmATLAS.yaml technique object with id AML.T0048.002 (pinned release v5.6.0) MITREOriginal
- EU AI Act Article 5(1)(a) Prohibited AI practicesArticle 5(1)(a); where the catalogue cites exploitation, compare Article 5(1)(b) European Union (EUR-Lex)Original
- EU AI Act Article 50(2), 50(4) Transparency obligations for providers and deployers of certain AI systemsArticle 50(2) and 50(4) European Union (EUR-Lex)Original
- EU AI Act Article 55(1)(b) Obligations of providers of general-purpose AI models with systemic riskArticle 55(1)(b) European Union (EUR-Lex)Original
- BSI R10 Erzeugung ver- und gefälschter Inhalte (Text, Bild, Video)Kap. 4, R10, p. 18 Bundesamt für Sicherheit in der Informationstechnik (BSI)Original
- BSI R5 Problematische und verzerrte Ausgaben (Text, Bild, Video)Kap. 4, R5, p. 16 Bundesamt für Sicherheit in der Informationstechnik (BSI)Original
Related threats
More entries from the topic group Malicious Use for Attacks, Fraud and Disinformation.
Assess this threat in your own system
The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.
Cite this entry
For reports, policies or internal documents; the link leads directly to this entry.
“Disinformation”. Versatile AI Risk Assessment, AI threat catalogue, as of July 2026. https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/disinformation/