Input data and information sources
Claims, applications, tickets and websites come from outsiders. Claims, applications and customer requests concern specific people.
AI agent
Also known as: AI agent, agentic AI
An AI agent breaks a task into steps and completes it largely independently using tools, interfaces and its own permissions in business applications.
Understand the use case
What defines this use case?
An insurer uses agents to prepare claims, German municipalities have trialled them, and a telecommunications company uses them in customer service.
This page describes a typical system design. Actual data flows and permissions depend on your implementation.
- System design
- Agent with tool access
- Ability to act
- Can act within applications
- Fields of use
- insurance, public administration, telecommunications
- Also known as
- agentic system, workflow agent, research agent, browser agent, computer-use agent
How it works, step by step
System design and data flows
Input data
A task arrives, such as a claim, application or ticket.
Claims, applications, tickets and websites come from outsiders. Agents controlling a browser or the whole screen analyse screen content as images.
Processing
A language model plans steps, calls tools and interfaces, and evaluates the responses.
The model breaks an instruction into steps and selects suitable calls from the approved tools. An execution environment runs these calls and returns the results to the model. This can lead to further steps. Planning, actual tool execution and approval are distinct parts of the system.
Results
The agent creates cases, edits records or presents a complete decision proposal to a person.
A generated result is newly worded or created. To assess it, the instruction and context on which it is based must remain identifiable.
Use and impact
Its actions change business applications immediately and can trigger payments, administrative decisions or customer responses.
Write or execution permissions allow the system to change a state within an application. Model responses, tool calls and changes actually executed must therefore be considered separately.
Explore the system design: Select a component to see its task, incoming and outgoing data, and role in the use case. The guided walkthrough explains each phase. Then use Yes, Uncertain and No to represent your specific system variant.
15 review first27 clarify9 baseline threats
- Present
- Depends on your system
- Absent
- Review first
- To clarify
How to use the model
The diagram shows the typical architecture. Select a component to see what happens there and adapt its features to your system. Enable “Analysis guidance” to see which threats need reviewing for this architecture. The page's checklist continues to describe the typical case.
Changes from the typical case
None yet. Each change will appear here.
Where most threats arise
- 2 review first, 13 clarify
- 6 review first
- 3 review first, 2 clarify
- 4 clarify
In this system design: 15 to review first, 27 to clarify, 9 baseline threats.
Architecture as a list
External access: Depends on your system
To clarify
Input: Part of every AI system
Review first
To clarify
Baseline for every AI system
Third-party content: Present
Review first
- Prompt Injection: Indirect Review priority
AI model: Part of every AI system
Review first
- Agentic AI / Autonomous Agents Review priority
- Sleepy Agent (Time/Event-Triggered Hidden Instructions)
- Prompt Injection: Direct
- Jailbreaks
- Meta Prompt Extraction
- Misalignment
Baseline for every AI system
Provider service: Present
Review first
To clarify
Document collection: Depends on your system
To clarify
Memory: Depends on your system
To clarify
Training and feedback: Depends on your system
To clarify
Other AI agents: Depends on your system
Tools and interfaces: Present
Review first
- Insecure Tool Design Review priority
- Excessive Agency Review priority
- Exfiltration from ML Application
To clarify
Result: Part of every AI system
Review first
To clarify
- Hate Speech and Discrimination
- Profanity
- Sexual Content
- Violence / Unsafe Actions
- Controversial Topics
- Illegal Activities
- Self-harm
- Harassment
- Unethical Actions
- Social Engineering
- Fraud
- Malicious Software
- Disinformation
Baseline for every AI system
Business application or equipment: Present
person: Part of every AI system
Baseline for every AI system
Affected person: Depends on your system
Unapproved AI services: Depends on your system
To clarify
Supply chain and technical foundations: 5 baseline threats applying to every AI system
Baseline for every AI system
How it differs from similar use cases
Edits and tests code in the repository
AI assistant in office applications
Helps a user in an office application without executing a sequence of steps independently
Customer service chatbots and voicebots
Answers requests in conversation and passes difficult cases to people
Data and information
Processing and operating environment
The language model controlling the agent generally comes from another provider.
Storage and further use
Research agents search document collections or the internet; others work only with business application data. Some agents store intermediate results and findings for reuse in later tasks.
Results and data sharing
The agent calls programs, interfaces and searches itself, such as weather or policy data in claims handling. It creates cases, edits records or prepares payments, changing business applications.
Using a provider does not automatically determine where processing takes place. Storage location, subcontractors, deletion periods and use for training must be checked for the specific deployment.
Assess risks and determine measures
From understanding the system to analysing risks
- Define the system boundary
Record data, participants, interfaces and permissions. Resolve uncertain characteristics in the model.
- Investigate threats
Relate review candidates to actual data flows: what could happen, and who would be affected?
- Assess risk
Assess impacts and likelihood in the deployment context. The number of review candidates is not a risk score.
- Provide evidence of measures
Select appropriate controls, assign responsibilities and define effectiveness checks. Document residual risks and approvals.
Key threats in the application context
- 4 particularly important
- 11 typical
- 27 depending on the system
- 9 for every AI system
A total of 51 threats from the threat catalogue may be relevant to this use case.
Priority review points
A crafted ticket or claim contains instructions the agent treats as a task. It executes them with its system permissions, for example disclosing data or approving a case.
The agent has write access across several business applications despite needing only read access. An error or injected instruction then changes records or triggers payments.
Agentic AI / Autonomous Agents
The agent chooses an unforeseen approach and executes it step by step. It has already processed many cases by the time someone reviews the results.
An agent's tool accepts arbitrary inputs without checking who initiated the call. The agent thus reaches data and functions unrelated to its task.
11 further review candidates in the typical design
Other relevant threats
Training data or an adopted base model can introduce hidden triggers for misbehaviour.
Listed because of this characteristic
- Uses another provider's model
Sleepy Agent (Time/Event-Triggered Hidden Instructions)
An instruction-following model can change behaviour at a particular time or event.
Listed because of this characteristic
- Performs natural language tasks
Users can override the system's working instructions through their inputs.
Listed because of this characteristic
- Performs natural language tasks
People requesting generated content may try to bypass intended limits.
Listed because of this characteristic
- Performs natural language tasks
Carefully crafted queries can reveal system instructions, which must therefore contain no secrets.
Listed because of this characteristic
- Performs natural language tasks
Complex or high-volume inputs can exhaust model computing resources.
Listed because of this characteristic
- Generates its own text
Targeted queries can recover some personal or protected training data from the model.
Listed because of these characteristics
- Processes personal data
- Processes confidential data
Sensitive Information Disclosure
Confidential content can appear in answers, supplied context or logs.
Listed because of these characteristics
- Processes personal data
- Processes confidential data
Exfiltration from ML Application
Protected data can leak through tools, network connections, storage or insiders.
Listed because of these characteristics
- Calls other programs or interfaces
- Processes confidential data
Factual Inconsistencies (Hallucinations)
Generated text can invent facts or distort them during rewriting.
Listed because of this characteristic
- Generates its own text
The target metric can improve while the underlying objective is missed.
Listed because of this characteristic
- Determines its own work steps
Up to 27 further threats may apply, depending on your system design. Nine threats need to be reviewed for every AI system.
System features and open questions
Each Yes adds further threats to your checklist, up to 27 in total. If you cannot yet answer a question, include its threats in your review.
- Added by this Yes
- Also requires a Yes to another question
How your system is built
- Generates content from open-ended requests10 threats, plus 2 only in combination with other characteristics
- Uses the Model Context Protocol (MCP)2 threats
- Uses operational feedback2 threats
- Searches its own document collection1 threat, plus 1 only in combination with other characteristics
- Learns from your own data1 threat, plus 1 only in combination with other characteristics
- Formulates answers from retrieved documents1 threat
- Remembers content beyond a single conversation1 threat
- Works with other AI agents1 threat
- Learns from examples with correct answers1 threat
- Processes text, images and speech together1 threat
- Generates code or queries1 threat
- One search store for several permission groups1 threat only in combination with other characteristics
How you operate the system
- Available to people outside the organisation4 threats, plus 2 only in combination with other characteristics
- Accepts queries directly through an interface4 threats
- Accessible over a network1 threat, plus 1 only in combination with other characteristics
- Incurs costs with each query1 threat
- Shares computing resources with other customers or applications1 threat
- Used alongside unapproved AI services1 threat
How your system is built
Does the system create new content from open-ended requests that people read, see or hear directly? Translation, reading aloud and transcription do not count.
An agent drafting responses to customers or citizens reaches people directly; purely background workflows do not.
If yes, add
Does the system search its own collection of documents or records for relevant content?
Research agents search document collections or the internet; others work only with business application data.
If yes, add
Does the system formulate answers from retrieved documents?
If yes, add
Does the system remember content from earlier conversations or tasks and reuse it later?
Some agents store intermediate results and findings for reuse in later tasks.
If yes, add
Does the system delegate tasks to other AI agents or incorporate their results?
At Allianz, seven specialised agents collaborate on each claim; simpler agents work alone.
If yes, add
Is the system connected to tools or data sources through the Model Context Protocol (MCP)?
Many agents connect tools and data sources through the Model Context Protocol (MCP).
If yes, add
Do documents from several customers, departments or permission groups share one search store?
If yes, add
Does your organisation train or fine-tune the model itself using its own data?
If yes, add
Is the model trained on examples to which someone has assigned the correct answer?
If yes, add
Does operational feedback or output later feed into analysis or training?
If yes, add
Does the system process several input types together, such as text with images or speech?
Agents controlling a browser or the whole screen analyse screen content as images.
If yes, add
Does the system generate code, scripts or database queries for further use?
If yes, add
How you operate the system
These questions arise for almost every AI system.
Can people or programs query the model directly, for example through an interface?
If yes, add
Is the system accessible over a network or dependent on network services?
If yes, add
Do you pay per query or unit of computing time, so costs increase with use?
Each step calls the language model again; a loop can rapidly increase costs.
If yes, add
Does the system share servers or computing resources with other customers or applications?
If yes, add
Can people outside your organisation use the system, including without login?
If yes, add
Do staff also use AI services for this task that your organisation has not approved?
If yes, add
Controls and implementation
Measures for the priority review areas are expanded. Each linked measure explains how to implement it.
Review priorityPrompt Injection: Indirect3 measures
Review priorityExcessive Agency3 measures
Review priorityAgentic AI / Autonomous Agents3 measures
Review priorityInsecure Tool Design3 measures
Backdoor ML Model2 measures
Sleepy Agent (Time/Event-Triggered Hidden Instructions)3 measures
Prompt Injection: Direct3 measures
Jailbreaks2 measures
Meta Prompt Extraction2 measures
Model Denial of Service3 measures
Privacy Attacks3 measures
Sensitive Information Disclosure3 measures
Exfiltration from ML Application3 measures
Factual Inconsistencies (Hallucinations)3 measures
Controls for the remaining threats are available on their pages in the threat catalogue.
AI Act provisions
-
Art. 4: AI literacy
For every AI system Applies since 2 February 2025.
-
Art. 53 para. 1 point b, Annex XII: Embedded general-purpose model
Depending on use Review trigger since 2 August 2025.
-
Art. 50 para. 1: Disclosure of AI interaction
In typical use Review trigger since 2 August 2026.
-
Annex III no. 5 point a: Public assistance benefits
Depending on use Review trigger from 2 December 2027.
Applies to every AI system
Art. 4: AI literacy
Applies if staff or appointed representatives operate or use AI systems.
Applies since 2 February 2025.
Obligations and details for Art. 4: AI literacy
- Who bears the obligations
- Provider, Deployer, Product manufacturer, Provider of a general-purpose AI model
Detailed obligations
May apply in typical use
Art. 50 para. 1: Disclosure of AI interaction
Applies if the agent communicates directly with customers or citizens and it is not obvious that an AI is responding.
Applies since 2 August 2026.
Obligations and details for Art. 50 para. 1: Disclosure of AI interaction
- The general condition is
- Applies if people directly write or speak with the system and it is not obvious that they are interacting with AI.
- Who bears the obligations
- Provider, Product manufacturer
Detailed obligations
May apply in particular uses
Annex III no. 5 point a: Public assistance benefits
Applies if the agent checks applications for public assistance benefits or prepares decisions on them for an authority.
Applies from 2 December 2027.
Obligations and details for Annex III no. 5 point a: Public assistance benefits
- The general condition is
- Applies if authorities or bodies acting for them use the system to assess eligibility for public benefits and services, or to grant, reduce or recover them.
- Who bears the obligations
- Provider, Deployer, Product manufacturer
Detailed obligations
Art. 53 para. 1 point b, Annex XII: Embedded general-purpose model
Applies if your organisation builds the agent itself and controls it with a large language model or another third-party general-purpose model.
Applies since 2 August 2025.
Obligations and details for Art. 53 para. 1 point b, Annex XII: Embedded general-purpose model
- The general condition is
- Applies if another provider's general-purpose AI model is incorporated, such as a large language model.
- Points to consider
- The model provider bears its obligations. It must give you the Annex XII information; use it for your own documentation.
- Who bears the obligations
- Provider of a general-purpose AI model
Detailed obligations
Classify your system with the quick checker All obligations in the obligations compass
Cross-cutting threats
Review these threats for every AI system, whatever its purpose.
Atypical system features
View 1 further question
If any of these features does apply to your system, add the listed threats.
Does the system use a knowledge graph storing concepts and their relationships?
If yes, add
Sources
- Report of a deployment
Allianz, The storm subsides, the AI agent cleans up, 3 November 2025 According to the deployer
Allianz states that seven specialised agents prepare claims in live operation. People decide on payments.
- Specialist study
OECD Artificial Intelligence Papers No 65, Agentic AI in organisations, 2026
The OECD cites agents in use, including KDDI customer service and the Federal Ministry for Digital Transformation and Government Modernisation's SPARK and Agentic AI Hub.
- Legislation, norm or standard
OWASP Top 10 for Agentic Applications 2026
OWASP treats agentic applications as a distinct system class with risks including goal hijacking, tool misuse and privilege abuse.
- Register or case collection
Crafted tickets make internal agents execute third-party instructions.
- Specialist study
Brave, Indirect Prompt Injection in Perplexity Comet
Researchers showed that an agentic browser could use injected instructions and the user's session permissions to capture an email address and one-time password.
Related use cases
Also in the field of Office work and knowledge management
- Generative AI assistant for staffStaff draft, revise, translate and shorten texts using a language model their organisation provides in a secured environment.
- AI assistant in office applicationsStaff invoke the assistant directly in email, word processing, spreadsheets and chat; it accesses mailboxes and file storage using their permissions.
- AI knowledge assistant (enterprise search)The assistant answers questions from a defined document collection and shows which documents support the answer.
- Speech recognition and transcriptionSpeech recognition automatically transcribes dictation, conversations and other recordings.
- AI meeting minutes and conversation documentationFrom recorded meetings and consultation conversations, the system drafts minutes or a case note with an action list.
- Machine translationPublic authorities and businesses automatically translate texts, websites and documents, often using their own glossary.
Often used together with
- AI-assisted insurance claims handlingThe system analyses claim documents and photographs, checks entitlement against the policy, flags inconsistencies and prepares settlement.
- Intelligent document processing (IDP)The system reads correspondence, invoices and forms, routes them to the responsible team and transfers their data into business systems.
Updated: 30 September 2026. Review checklist rules version 0.3.2, threat catalogue version 2026.10.24.2. How these pages are structured