Versatile AI Risk Assessment

AI threat catalogue Agentic and Autonomous AI Production

Agentic AI / Autonomous Agents

AI agents plan multi-step tasks on their own and carry them out with tools such as email, databases, or code execution. The less human involvement there is, the further planning errors, manipulated content, and knock-on failures spread.

As of: October 2026 · Catalogue version 2026.10.24.2 · 7 mitigations · 14 verified sources

Description

An AI agent breaks a task into individual steps, autonomously calls tools and interfaces, and feeds the results into the next step. Several risks compound along this chain: planning and judgment errors propagate from step to step, and attackers can redirect the agent through content it reads while working, for example via prompt injection (hidden instructions in emails, documents, or web pages). A redirected or misguided agent then uses its legitimate tools for harmful actions, executes code, or sends data outside the organization. In systems of multiple agents, a single fault can cascade across further agents and workflows.

Possible impact

Everything the agent can reach is exposed: unwanted transactions, data leakage, deleted files, and outages of entire workflows. Cascading effects can spread the damage beyond a single system. For high-risk systems, the EU AI Act requires that humans can effectively oversee, intervene in, and halt the system; for large general-purpose AI models, the regulation names the degree of autonomy and access to tools among the factors influencing systemic risks.

Example

A development agent with command-line access works through tasks from project files. A crafted file contains hidden instructions that the agent treats as an assignment: it runs a delete command and destroys data on the system. Throughout, it stays entirely within its regular tool permissions.

Recommended mitigations (7)

Compare effects and implementation levels. Expand each entry for the additional classification and rationale.

  • Strict scope limitation of agent actions

    Technical
    Effect
    Preventive, Impact-limiting
    Implementation level
    Application, API & agents
    Classification and rationale

    Primarily technical: Machine-enforced identity, permission, or scope rules constrain unauthorized access and actions.

  • Human approval gates for critical steps

    Organizational & process-based
    Effect
    Preventive, Impact-limiting
    Implementation level
    Application, API & agents, Use & operations
    Classification and rationale

    Complementary control typePeople & competence, Technical

    Primarily organizational and process-based: A binding approval gate stops critical agent actions before execution; human judgment decides whether to approve or reject them.

  • Action budgets and step limits

    Technical
    Effect
    Preventive, Impact-limiting
    Implementation level
    Application, API & agents
    Classification and rationale

    Primarily technical: Automated resource controls, budget limits, or runtime boundaries constrain overload, abuse, cost, and cascading failures.

  • Comprehensive agent observability

    Technical
    Effect
    Detective
    Implementation level
    Application, API & agents, Use & operations
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

  • Reversible operations where possible

    Technical
    Effect
    Impact-limiting, Restorative
    Implementation level
    Application, API & agents, Use & operations
    Classification and rationale

    Primarily technical: Versioned states or rollback mechanisms enable controlled recovery.

  • Simulation / dry-run before production

    Technical
    Effect
    Preventive, Detective
    Implementation level
    Application, API & agents, Use & operations
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

  • Agent security sandboxing

    Technical
    Effect
    Preventive, Impact-limiting
    Implementation level
    Application, API & agents, Infrastructure
    Classification and rationale

    Primarily technical: Architectural and runtime boundaries separate data, execution, or tenants and limit the spread of failures.

Framework mappings

Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.

OWASP LLM Top 10 ASI01:2026 · ASI02:2026 · ASI08:2026 · LLM01:2026 · LLM03:2026 · LLM06:2026NIST AI RMF Section 3.5MITRE ATLAS AML.T0051EU AI Act Article 14(4)(d) · Article 26(5) · Article 55(1)(a) · Article 9(1), 9(2)(a), 9(2)(d)BSI R28BIML BIML78 system:5

Verified references (14)

Every reference states the framework, the exact location and the publishing organisation.

EU AI Act obligations related to this threat

The obligations from the obligations compass whose articles the catalogue maps to this threat. Each leads to its entry with core, action step and evidence; whether it applies to you depends on role and risk class.

  • Establish and operate a risk management system Art. 9 · applies from Annex III: 2 December 2027 · Annex I: 2 August 2028

    Statutory roles: Provider, Product manufacturer. Classes: High-risk under Annex III, High-risk as a product or safety component.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Build in effective human oversight Art. 14 · applies from Annex III: 2 December 2027 · Annex I: 2 August 2028

    Statutory roles: Provider, Product manufacturer. Classes: High-risk under Annex III, High-risk as a product or safety component.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Monitor operation and suspend on risk Art. 26 Abs. 5 · applies from Annex III: 2 December 2027 · Annex I: 2 August 2028

    Statutory roles: Deployer. Classes: High-risk under Annex III, High-risk as a product or safety component.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Evaluate and mitigate systemic risks Art. 55 · applies from 2 August 2025

    Statutory roles: Provider of a general-purpose AI model. Classes: Model with systemic risk.

    This reference is a review prompt. A threat match alone does not establish applicability.

All 62 obligations by role, risk class and deadline in the obligations compass

Terms on this page

Glossary terms that occur in this entry. Every link leads to the full explanation.

More entries from the topic group Agentic and Autonomous AI.

Excessive Agency An AI system is granted more permissions or freedom to act than its task requires. Misalignment The AI system does not pursue the goals its operator or users intend but whatever it was actually optimized for. Agent Memory Poisoning (Persistent Context) AI agents with persistent memory carry information from earlier sessions forward as trusted context for future decisions.

Matching use cases

This list shows where to review this threat, rather than how high the risk is.

Particularly important for:

  • AI agentThe agent chooses an unforeseen approach and executes it step by step. It has already processed many cases by the time someone reviews the results.

Typical for:

For 2 further use cases, relevance depends on the system design, for example whether it determines its own work steps. Show these use cases

Frequently asked questions

What is Agentic AI / Autonomous Agents?
AI agents plan multi-step tasks on their own and carry them out with tools such as email, databases, or code execution. The less human involvement there is, the further planning errors, manipulated content, and knock-on failures spread.
Which mitigations help against Agentic AI / Autonomous Agents?
The catalogue lists 7 mitigations, each with control type, effect and implementation level. First among them are Strict scope limitation of agent actions, Human approval gates for critical steps and Action budgets and step limits.
Which EU AI Act obligations does Agentic AI / Autonomous Agents touch?
The catalogue maps this threat to Art. 9, Art. 14, Art. 26 Abs. 5 and Art. 55 of the EU AI Act. In the obligations compass this leads to: Establish and operate a risk management system, Build in effective human oversight, Monitor operation and suspend on risk and Evaluate and mitigate systemic risks. Whether an obligation applies depends on role, risk class and deadline.

Assess this threat in your own system

The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.

Start the live demo See the free modules

Cite this entry

For reports, policies or internal documents; the link leads directly to this entry.

“Agentic AI / Autonomous Agents”. Versatile AI Risk Assessment, AI threat catalogue, as of October 2026.
https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/autonomous-agents/

← Back to the full catalogue