Versatile AI Risk Assessment
Documents and casework

Intelligent document processing (IDP)

The system reads correspondence, invoices and forms, routes them to the responsible team and transfers their data into business systems.

Explore the interactive model

Export handout

A concise PDF covering how the system works, its data flows and points to investigate, ready to share.

The explanations and measures describe the typical system design. Extensive personal changes add an appendix.

Understand the use case

What defines this use case?

Public authorities and an insurer use such systems for incoming mail, applications and invoices.

This page describes a typical system design. Actual data flows and permissions depend on your implementation.

System design
Analysis and processing
Ability to act
Action permissions depend on integration
Fields of use
public administration, insurance, accounting
Also known as
incoming mail processing, receipt capture, incoming document classification, OCR with data extraction

How it works, step by step

System design and data flows

  1. Input data

    Correspondence, invoices and forms arrive as scans, PDFs or email attachments.

    Outsiders write correspondence, invoices and applications. Layout models analyse a scan's image and text together; basic OCR does not.

  2. Processing

    OCR reads the document; a machine learning model determines its type and extracts the required fields.

    Depending on the document type, processing combines text recognition, classification and extraction of individual fields. An invoice amount, for example, is assigned to a defined data field. Subsequent validation rules can check completeness and format; transferring the data into a business application is a separate integration step.

  3. Results

    The document type and fields enter the system as a record, with uncertain information flagged.

    The result describes the defined model task. A match, category or forecast acquires its professional significance only in conjunction with the underlying data and intended use.

  4. Use and impact

    The record goes to the responsible team or business system and can prepare payments, deadlines or administrative decisions.

    Integration determines whether the result is simply displayed, adopted after approval or written directly into a business application. These variants differ in their data flows and permissions.

Explore the system design: Select a component to see its task, incoming and outgoing data, and role in the use case. The guided walkthrough explains each phase. Then use Yes, Uncertain and No to represent your specific system variant.

8 review first16 clarify6 baseline threats

InputProcessingResultImpact Externalaccess 05 Input 41 Third-partycontent 00 AI model 05 Providerservice 11 Documentcollection 00 Memory 00 Training andfeedback 11 OtherAI agents 00 Tools andinterfaces 00 Result 11 Business appor equipment 01 person 10 Affectedperson 00 AI serviceswithout approval 01 Externalaccess 05 Input 41 Third-partycontent 00 AI model 05 Providerservice 11 Documentcollection 00 Memory 00 Training andfeedback 11 OtherAI agents 00 Tools andinterfaces 00 Result 11 Business appor equipment 01 person 10 Affectedperson 00 AI serviceswithout approval 01
  • Present
  • Depends on your system
  • Absent
  • Review first
  • To clarify
Architecture as a list

How it differs from similar use cases

Data and information

Input data and information sources

Outsiders write correspondence, invoices and applications. Applications, contracts and letters include names, addresses and case references.

Processing and operating environment

OCR and extraction models generally come from a provider.

Storage and further use

Some deployers further train recognition on their own documents; others use the model unchanged. Caseworkers' corrections can feed back as new training examples.

Results and data sharing

Depending on integration, the system creates records in the business system itself or submits them for approval. With a cloud service, documents leave the organisation's environment.

Using a provider does not automatically determine where processing takes place. Storage location, subcontractors, deletion periods and use for training must be checked for the specific deployment.

Assess risks and determine measures

From understanding the system to analysing risks

  1. Define the system boundary

    Record data, participants, interfaces and permissions. Resolve uncertain characteristics in the model.

  2. Investigate threats

    Relate review candidates to actual data flows: what could happen, and who would be affected?

  3. Assess risk

    Assess impacts and likelihood in the deployment context. The number of review candidates is not a risk score.

  4. Provide evidence of measures

    Select appropriate controls, assign responsibilities and define effectiveness checks. Document residual risks and approvals.

Key threats in the application context

  • 3 particularly important
  • 5 typical
  • 16 depending on the system
  • 6 additional threats for every AI system

A total of 30 threats from the threat catalogue may be relevant to this use case.

Priority review points

  1. Adversarial Inputs

    Submitters control document content. A crafted invoice can introduce false bank details into a record.

  2. Insecure Output Handling

    Extracted fields go straight into the business system. Accepting them unchecked lets incorrect amounts or references trigger payments or deadlines.

  3. Overreliance

    Approving many extracted records every day can become a cursory task. Wrong amounts or allocations then go unnoticed.

5 further review candidates in the typical design

Other relevant threats

  1. Targeted Poisoning / Label Poisoning

    Falsified training labels can deliberately shift model outputs.

    Listed because of this characteristic

    • Learns from examples with correct answers

  2. Backdoor ML Model

    Training data or an adopted base model can introduce hidden triggers for misbehaviour.

    Listed because of this characteristic

    • Uses another provider's model

  3. Privacy Attacks

    Targeted queries can recover some personal or protected training data from the model.

    Listed because of these characteristics

    • Processes personal data
    • Processes confidential data

  4. Sensitive Information Disclosure

    Confidential content can appear in answers, supplied context or logs.

    Listed because of these characteristics

    • Processes personal data
    • Processes confidential data

  5. Exfiltration from ML Application

    Protected data can leak through tools, network connections, storage or insiders.

    Listed because of this characteristic

    • Processes confidential data

Up to 16 further threats may apply, depending on your system design. Nine threats need to be reviewed for every AI system. Three of these are particularly important here and are therefore listed above.

System features and open questions

Each Yes adds further threats to your checklist, up to 16 in total. If you cannot yet answer a question, include its threats in your review.

How many threats a Yes adds
  • Added by this Yes
  • Also requires a Yes to another question

How your system is built

  1. Performs natural language tasks5 threats
  2. Generates its own text2 threats, plus 1 only in combination with other characteristics
  3. Processes text, images and speech together2 threats
  4. Learns from your own data1 threat, plus 1 only in combination with other characteristics
  5. Changes data or triggers processes1 threat
  6. Uses operational feedback1 threat

How you operate the system

  1. Available to people outside the organisation5 threats
  2. Accepts queries directly through an interface4 threats, plus 1 only in combination with other characteristics
  3. Accessible over a network2 threats, plus 1 only in combination with other characteristics
  4. Incurs costs with each query1 threat
  5. Shares computing resources with other customers or applications1 threat
  6. Used alongside unapproved AI services1 threat

How your system is built

Can the system change things itself, such as storing data, sending messages, initiating transactions or granting permissions?

Depending on integration, the system creates records in the business system itself or submits them for approval.

If yes, add

Does your organisation train or fine-tune the model itself using its own data?

Some deployers further train recognition on their own documents; others use the model unchanged.

If yes, add

Does operational feedback or output later feed into analysis or training?

Caseworkers' corrections can feed back as new training examples.

If yes, add

Does the system process several input types together, such as text with images or speech?

Layout models analyse a scan's image and text together; basic OCR does not.

If yes, add

How you operate the system

These questions arise for almost every AI system.

Controls and implementation

Measures for the priority review areas are expanded. Each linked measure explains how to implement it.

Review priorityAdversarial Inputs2 measures
Review priorityInsecure Output Handling3 measures
Review priorityOverreliance3 measures
Targeted Poisoning / Label Poisoning3 measures
Backdoor ML Model2 measures
Privacy Attacks3 measures
Sensitive Information Disclosure3 measures
Exfiltration from ML Application3 measures

Controls for the remaining threats are available on their pages in the threat catalogue.

AI Act provisions

When AI Act provisions apply
  1. Art. 4: AI literacy

    For every AI system Applies since 2 February 2025.

  2. Art. 53 para. 1 point b, Annex XII: Embedded general-purpose model

    Depending on use Review trigger since 2 August 2025.

  3. Annex III no. 5 point a: Public assistance benefits

    Depending on use Review trigger from 2 December 2027.

  4. Annex III no. 5 point b: Creditworthiness

    Depending on use Review trigger from 2 December 2027.

Applies to every AI system

Art. 4: AI literacy

Applies if staff or appointed representatives operate or use AI systems.

Applies since 2 February 2025.

Obligations and details
Who bears the obligations
Provider, Deployer, Product manufacturer, Provider of a general-purpose AI model

Detailed obligations

May apply in particular uses

Annex III no. 5 point a: Public assistance benefits

Applies if an authority uses the extracted information to decide on public benefits, for example at a job centre.

Applies from 2 December 2027.

Obligations and details
The general condition is
Applies if authorities or bodies acting for them use the system to assess eligibility for public benefits and services, or to grant, reduce or recover them.
Who bears the obligations
Provider, Deployer, Product manufacturer

Detailed obligations

Annex III no. 5 point b: Creditworthiness

Applies if extracted documents feed directly into assessing a person's creditworthiness.

Applies from 2 December 2027.

Obligations and details
The general condition is
Applies if the system assesses people's creditworthiness or establishes their credit scores.
Exception
Systems intended to detect financial fraud.
Who bears the obligations
Provider, Deployer, Product manufacturer

Detailed obligations

Art. 53 para. 1 point b, Annex XII: Embedded general-purpose model

Applies if a large language model or another third-party general-purpose model extracts the information.

Applies since 2 August 2025.

Obligations and details
The general condition is
Applies if another provider's general-purpose AI model is incorporated, such as a large language model.
Points to consider
The model provider bears its obligations. It must give you the Annex XII information; use it for your own documentation.
Who bears the obligations
Provider of a general-purpose AI model

Detailed obligations

Generally does not apply

Art. 50 para. 2: Marking artificially generated content

The system only reproduces information from existing documents and creates no new content. This generally also applies when a language model extracts fields.

Obligations and details
The general condition is
Applies if the system generates synthetic text, images, audio or video.
Exception
Exceptions include ordinary editing assistance that does not substantially alter inputs or their meaning.
Points to consider
Systems marketed before 2 August 2026 must be updated by 2 December 2026.
Who bears the obligations
Provider, Product manufacturer, Provider of a general-purpose AI model

Detailed obligations

Classify your system with the quick checker All obligations in the obligations compass

Cross-cutting threats

Review these threats for every AI system, whatever its purpose.

Why these threats always belong on the checklist

Atypical system features

View 14 further questions

If any of these features does apply to your system, add the listed threats.

Sources

Related use cases

Also in the field of Documents and casework

Often used together with

Updated: 30 September 2026. Review checklist rules version 0.3.2, threat catalogue version 2026.10.24.2. How these pages are structured