AI threat catalogueAgentic and Autonomous AIProduction
Agentic AI / Autonomous Agents
AI agents plan multi-step tasks on their own and carry them out with tools such as email, databases, or code execution. The less human involvement there is, the further planning errors, manipulated content, and knock-on failures spread.
Description
An AI agent breaks a task into individual steps, autonomously calls tools and interfaces, and feeds the results into the next step. Several risks compound along this chain: planning and judgment errors propagate from step to step, and attackers can redirect the agent through content it reads while working, for example via prompt injection (hidden instructions in emails, documents, or web pages). A redirected or misguided agent then uses its legitimate tools for harmful actions, executes code, or sends data outside the organization. In systems of multiple agents, a single fault can cascade across further agents and workflows.
Possible impact
Everything the agent can reach is exposed: unwanted transactions, data leakage, deleted files, and outages of entire workflows. Cascading effects can spread the damage beyond a single system. For high-risk systems, the EU AI Act requires that humans can effectively oversee, intervene in, and halt the system; for large general-purpose AI models, the regulation names the degree of autonomy and access to tools among the factors influencing systemic risks.
Example
A development agent with command-line access works through tasks from project files. A crafted file contains hidden instructions that the agent treats as an assignment: it runs a delete command and destroys data on the system. Throughout, it stays entirely within its regular tool permissions.
Recommended mitigations (7)
Every mitigation states its control type, effect, implementation level and the reason for the classification.
Strict scope limitation of agent actionsTechnical
- Effect
- Preventive, Impact-limiting
- Implementation level
- Application, API & agents
- Reason for the classification
- “Strict scope limitation of agent actions” is primarily technical: Machine-enforced identity, permission, or scope rules constrain unauthorized access and actions.
Human approval gates for critical stepsOrganizational & process-based
- Effect
- Preventive, Impact-limiting
- Implementation level
- Application, API & agents, Use & operations
- Complementary control type
- People & competence, Technical
- Reason for the classification
- “Human approval gates for critical steps” is primarily organizational and process-based: A binding approval gate stops critical agent actions before execution; human judgment decides whether to approve or reject them.
Action budgets and step limitsTechnical
- Effect
- Preventive, Impact-limiting
- Implementation level
- Application, API & agents
- Reason for the classification
- “Action budgets and step limits” is primarily technical: Automated resource controls, budget limits, or runtime boundaries constrain overload, abuse, cost, and cascading failures.
Comprehensive agent observabilityTechnical
- Effect
- Detective
- Implementation level
- Application, API & agents, Use & operations
- Reason for the classification
- “Comprehensive agent observability” is primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.
Reversible operations where possibleTechnical
- Effect
- Impact-limiting, Restorative
- Implementation level
- Application, API & agents, Use & operations
- Reason for the classification
- “Reversible operations where possible” is primarily technical: Versioned states or rollback mechanisms enable controlled recovery.
Simulation / dry-run before productionTechnical
- Effect
- Preventive, Detective
- Implementation level
- Application, API & agents, Use & operations
- Reason for the classification
- “Simulation / dry-run before production” is primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.
Agent security sandboxingTechnical
- Effect
- Preventive, Impact-limiting
- Implementation level
- Application, API & agents, Infrastructure
- Reason for the classification
- “Agent security sandboxing” is primarily technical: Architectural and runtime boundaries separate data, execution, or tenants and limit the spread of failures.
Framework mappings
Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.
Verified references (13)
Every reference states the framework, the exact location and the publishing organisation.
- OWASP LLM Top 10 ASI01:2026 Agent Goal HijackASI01 Agent Goal Hijack, pp. 9–11 of the official PDF OWASP FoundationOriginal
- OWASP LLM Top 10 ASI02:2026 Tool Misuse and ExploitationASI02 Tool Misuse and Exploitation, pp. 12–14 of the official PDF OWASP FoundationOriginal
- OWASP LLM Top 10 ASI08:2026 Cascading FailuresASI08 Cascading Failures, pp. 30–32 of the official PDF OWASP FoundationOriginal
- OWASP LLM Top 10 LLM01:2025 Prompt InjectionLLM01:2025 Prompt Injection, official category page OWASP FoundationOriginal
- OWASP LLM Top 10 LLM06:2025 Excessive AgencyLLM06:2025 Excessive Agency, official category page OWASP FoundationOriginal
- NIST AI RMF Section 3.5 Security of AgentsSection 3.5, p. 54 National Institute of Standards and Technology (NIST)Original
- MITRE ATLAS AML.T0051 LLM Prompt InjectionATLAS.yaml technique object with id AML.T0051 (pinned release v5.6.0) MITREOriginal
- EU AI Act Article 14(4)(d) Human oversightArticle 14(4)(d); for automation bias, Article 14(4)(b) European Union (EUR-Lex)Original
- EU AI Act Article 26(5) Obligations of deployers of high-risk AI systemsArticle 26(5) European Union (EUR-Lex)Original
- EU AI Act Article 55(1)(a) Obligations of providers of general-purpose AI models with systemic riskArticle 55(1)(a) European Union (EUR-Lex)Original
- EU AI Act Article 9(1), 9(2)(a), 9(2)(d) Risk management systemArticle 9(1), 9(2)(a), 9(2)(d), read with Article 9(3) European Union (EUR-Lex)Original
- BSI R28 Indirect Prompt Injections (Text)Kap. 4, R28, p. 33 Bundesamt für Sicherheit in der Informationstechnik (BSI)Original
- BIML BIML78 system:5 Error PropagationPDF p. 25, [system:5:error propagation] Berryville Institute of Machine Learning (BIML)Original
Terms on this page
Glossary terms that occur in this entry. Every link leads to the full explanation.
- Prompt Injection Manipulated input or planted content redirects a language model.
- Agentic system (agentic AI) A system that uses a language model to carry out multi-step tasks on its own.
- Degree of autonomy How much a system decides and executes without human involvement.
Related threats
More entries from the topic group Agentic and Autonomous AI.
Assess this threat in your own system
The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.
Cite this entry
For reports, policies or internal documents; the link leads directly to this entry.
“Agentic AI / Autonomous Agents”. Versatile AI Risk Assessment, AI threat catalogue, as of July 2026. https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/autonomous-agents/