Versatile AI Risk Assessment

AI threat catalogue Malicious Use for Attacks, Fraud and Disinformation Production

Social Engineering

Attackers use AI to produce deceptively authentic, personally tailored scam messages, calls and pretext stories at scale. Familiar warning signs such as clumsy language disappear, making the deception considerably more convincing.

As of: October 2026 · Catalogue version 2026.10.24.2 · 5 mitigations · 7 verified sources

Description

Social engineering tricks people into revealing confidential information, making payments or installing malware. Generative AI amplifies this tactic considerably: language models write flawless phishing messages (fake communications designed to prompt a harmful action) tailored to individual recipients and their company, and provide scripts for fraudulent phone calls. Voice and video generators additionally imitate real people, such as managers or business partners. The attacks target people rather than technology, arriving by email, phone, messenger or video call. AI lowers the entry barrier and increases the volume, speed and quality of such attacks.

Possible impact

A successful deception can lead to fraudulent payments, stolen credentials and, in turn, compromised systems and data leaks. Beyond the financial damage, reporting and liability questions arise, for example when personal data is exposed. Staff in finance, HR and support roles are particularly at risk, and the trust of customers and partners in the company’s communication suffers as well.

Example

The accounting team receives an email that precisely matches the tone and writing style of the CEO; shortly afterwards a call arrives using a cloned version of the CEO’s voice: a supposedly confidential acquisition requires an immediate transfer. Attacks of this kind, known as CEO fraud, become far more convincing with AI-generated text and voices.

Recommended mitigations (5)

Compare effects and implementation levels. Expand each entry for the additional classification and rationale.

  • Detection of impersonation/pretext patterns

    Technical
    Effect
    Detective
    Implementation level
    Application, API & agents
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

  • Refusal for deceptive content generation

    Technical
    Effect
    Preventive
    Implementation level
    Application, API & agents
    Classification and rationale

    Primarily technical: System-enforced inspection, transformation, or blocking rules stop or neutralize disallowed content before further processing.

  • Identity verification in sensitive workflows

    Technical
    Effect
    Preventive, Detective
    Implementation level
    Application, API & agents, Use & operations
    Classification and rationale

    Complementary control typeOrganizational & process-based

    Primarily technical: Machine-enforced identity, permission, or scope rules constrain unauthorized access and actions; complemented by binding workflows.

  • User awareness training

    People & competence
    Effect
    Preventive
    Implementation level
    Organization, Use & operations
    Classification and rationale

    Complementary control typeOrganizational & process-based

    Primarily a people-and-competence control: Trained users recognize deception and abuse patterns; audience, repetition, and effectiveness requirements support application.

  • Anti-phishing detection

    Technical
    Effect
    Detective
    Implementation level
    Application, API & agents
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

Framework mappings

Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.

NIST AI RMF Section 2.8 · Section 2.9MITRE ATLAS AML.T0048.002EU AI Act Article 5(1)(a) · Article 55(1)(b) · Article 9(1), 9(2)(a), 9(2)(d)BSI R11

Verified references (7)

Every reference states the framework, the exact location and the publishing organisation.

EU AI Act obligations related to this threat

The obligations from the obligations compass whose articles the catalogue maps to this threat. Each leads to its entry with core, action step and evidence; whether it applies to you depends on role and risk class.

  • Refrain from the prohibited practices Art. 5 Abs. 1 · applies from 2 February 2025

    Statutory roles: Provider, Deployer, Importer, Distributor, Authorised representative, Product manufacturer, Provider of a general-purpose AI model. Classes: Prohibited practice.

    These prohibitions have applied since 2 February 2025 and are unaffected by the deferral of the high-risk obligations. Infringements carry the highest fine bracket of the regulation.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • New prohibition: intimate depictions without consent Art. 5 Abs. 1 UAbs. 1 Buchst. ba, Art. 5 Abs. 1a, 1b · applies from 2 December 2026 · amended by Reg. 2026/1744

    Statutory roles: Provider, Deployer, Importer, Distributor, Authorised representative, Product manufacturer, Provider of a general-purpose AI model. Classes: Prohibited practice, System subject to transparency obligations.

    This prohibition is new and was inserted by Regulation (EU) 2026/1744. It applies from 2 December 2026. For deployers, the prohibition applies only where the system is used for that very purpose.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • New prohibition: child sexual abuse material Art. 5 Abs. 1 UAbs. 1 Buchst. bb · applies from 2 December 2026 · amended by Reg. 2026/1744

    Statutory roles: Provider, Deployer, Importer, Distributor, Authorised representative, Product manufacturer, Provider of a general-purpose AI model. Classes: Prohibited practice.

    Also newly inserted by Regulation (EU) 2026/1744, applying from 2 December 2026.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Establish and operate a risk management system Art. 9 · applies from Annex III: 2 December 2027 · Annex I: 2 August 2028

    Statutory roles: Provider, Product manufacturer. Classes: High-risk under Annex III, High-risk as a product or safety component.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Evaluate and mitigate systemic risks Art. 55 · applies from 2 August 2025

    Statutory roles: Provider of a general-purpose AI model. Classes: Model with systemic risk.

    This reference is a review prompt. A threat match alone does not establish applicability.

All 62 obligations by role, risk class and deadline in the obligations compass

More entries from the topic group Malicious Use for Attacks, Fraud and Disinformation.

Fraud Criminals use AI to commit fraud at greater scale and with greater credibility: through fake reviews, invented identities, forged … Malicious Software Attackers use AI models to generate or improve working malware and attack tooling. Disinformation AI makes it possible to mass-produce false or misleading content of convincing quality: fabricated news articles, manipulated images and …

Matching use cases

This list shows where to review this threat, rather than how high the risk is.

Typical for:

For 6 further use cases, relevance depends on the system design, for example whether it generates new content from open-ended requests or generates its own text. Show these use cases

Frequently asked questions

What is Social Engineering?
Attackers use AI to produce deceptively authentic, personally tailored scam messages, calls and pretext stories at scale. Familiar warning signs such as clumsy language disappear, making the deception considerably more convincing.
Which mitigations help against Social Engineering?
The catalogue lists 5 mitigations, each with control type, effect and implementation level. First among them are Detection of impersonation/pretext patterns, Refusal for deceptive content generation and Identity verification in sensitive workflows.
Which EU AI Act obligations does Social Engineering touch?
The catalogue maps this threat to Art. 5 Abs. 1, Art. 5 Abs. 1 UAbs. 1 Buchst. ba, Art. 5 Abs. 1 UAbs. 1 Buchst. bb, Art. 9 and Art. 55 of the EU AI Act. In the obligations compass this leads to: Refrain from the prohibited practices, New prohibition: intimate depictions without consent, New prohibition: child sexual abuse material, Establish and operate a risk management system and Evaluate and mitigate systemic risks. Whether an obligation applies depends on role, risk class and deadline.

Assess this threat in your own system

The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.

Start the live demo See the free modules

Cite this entry

For reports, policies or internal documents; the link leads directly to this entry.

“Social Engineering”. Versatile AI Risk Assessment, AI threat catalogue, as of October 2026.
https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/social-engineering/

← Back to the full catalogue