AI threat catalogueMalicious Use for Attacks, Fraud and DisinformationProduction
Fraud
Criminals use AI to commit fraud at greater scale and with greater credibility: through fake reviews, invented identities, forged documents, and cloned voices and videos.
Description
Generative AI gives fraudsters the building blocks of a credible deception. Language models write convincing scam messages and fake product reviews; image generators create profile photos, identity documents and entire invented identities used to set up fictitious accounts and social media profiles in bulk. Voice clones and deepfakes (AI-generated image, audio or video content that convincingly resembles real people) impersonate relatives, business partners or well-known figures to trigger payments. Points of attack include payment and ordering processes, customer channels, and identity checks such as video identification, whose biometric controls can be defeated with deepfakes.
Possible impact
Companies face direct financial losses from fraudulently obtained payments and accounts opened under false identities. Fake reviews and fraudulent appearances in the company’s name damage the brand and customer trust. Where identity checks are defeated, legal and regulatory risks follow, for example around anti-money-laundering and customer identification duties (know your customer).
Example
Fraudsters open accounts with a financial services provider using AI-generated ID photos and pass the video identification with a deepfake face injected in real time. They then route fraudulent payments through the accounts opened this way.
Recommended mitigations (5)
Every mitigation states its control type, effect, implementation level and the reason for the classification.
Fraud detection models in workflowsTechnical
- Effect
- Detective
- Implementation level
- Model & training, Application, API & agents
- Reason for the classification
- “Fraud detection models in workflows” is primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.
Identity and document verificationTechnical
- Effect
- Preventive, Detective
- Implementation level
- Application, API & agents, Use & operations
- Complementary control type
- Organizational & process-based
- Reason for the classification
- “Identity and document verification” is primarily technical: Machine-enforced identity, permission, or scope rules constrain unauthorized access and actions; complemented by binding workflows.
Anomaly detection in transactionsTechnical
- Effect
- Detective
- Implementation level
- Application, API & agents, Use & operations
- Reason for the classification
- “Anomaly detection in transactions” is primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.
Refusal for fraud-facilitating requestsTechnical
- Effect
- Preventive
- Implementation level
- Application, API & agents
- Reason for the classification
- “Refusal for fraud-facilitating requests” is primarily technical: System-enforced inspection, transformation, or blocking rules stop or neutralize disallowed content before further processing.
Forensic audit trailsTechnical
- Effect
- Detective
- Implementation level
- Application, API & agents, Use & operations
- Complementary control type
- Organizational & process-based
- Reason for the classification
- “Forensic audit trails” is primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators; complemented by binding workflows.
Framework mappings
Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.
Verified references (6)
Every reference states the framework, the exact location and the publishing organisation.
- NIST AI RMF Section 2.8 Information IntegritySection 2.8, pp. 9–10 National Institute of Standards and Technology (NIST)Original
- MITRE ATLAS AML.T0048.002 Societal HarmATLAS.yaml technique object with id AML.T0048.002 (pinned release v5.6.0) MITREOriginal
- EU AI Act Article 5(1)(a) Prohibited AI practicesArticle 5(1)(a); where the catalogue cites exploitation, compare Article 5(1)(b) European Union (EUR-Lex)Original
- EU AI Act Article 55(1)(b) Obligations of providers of general-purpose AI models with systemic riskArticle 55(1)(b) European Union (EUR-Lex)Original
- BSI R10 Erzeugung ver- und gefälschter Inhalte (Text, Bild, Video)Kap. 4, R10, p. 18 Bundesamt für Sicherheit in der Informationstechnik (BSI)Original
- BSI R11 Vortäuschen einer (medialen) Identität (Text, Bild, Video)Kap. 4, R11, p. 19 Bundesamt für Sicherheit in der Informationstechnik (BSI)Original
Related threats
More entries from the topic group Malicious Use for Attacks, Fraud and Disinformation.
Assess this threat in your own system
The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.
Cite this entry
For reports, policies or internal documents; the link leads directly to this entry.
“Fraud”. Versatile AI Risk Assessment, AI threat catalogue, as of July 2026. https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/fraud/