Input data and information sources
Images from other practices or hospitals do not originate from the organisation's own equipment. Each image belongs to a patient and carries a name, date of birth and examination data.
AI-assisted medical image analysis
Also known as: AI in medical imaging
The system marks abnormalities on X-rays, CT, MRI or mammography images to help clinicians interpret findings.
Understand the use case
What defines this use case?
The model learns from images with clinically confirmed findings, then marks abnormalities, classifies images or prioritises worklists. These systems operate in German mammography screening; hundreds are authorised as radiology medical devices in the US.
This page describes a typical system design. Actual data flows and permissions depend on your implementation.
- System design
- Analysis and processing
- Ability to act
- Action permissions depend on integration
- Fields of use
- healthcare
- Also known as
- radiology AI, AI diagnostic support, AI in mammography screening
How it works, step by step
System design and data flows
Input data
X-rays, CT, MRI or mammography images arrive from the hospital or practice image archive.
Images from other practices or hospitals do not originate from the organisation's own equipment. Some systems use earlier images or clinical information alongside the current image.
Processing
An image model searches for patterns learned from images with confirmed findings.
The model processes image data to produce annotations, segmentations or indications of unusual structures. These results relate to the selected imaging task and the model used. Their interpretation within an individual patient's findings takes place in the subsequent clinical workflow.
Results
Abnormal regions are marked, with a classification or worklist position.
The result describes the defined model task. A match, category or forecast acquires its professional significance only in conjunction with the underlying data and intended use.
Use and impact
The clinician uses this support to interpret findings and decide on further examination or treatment.
Integration determines whether the result is simply displayed, adopted after approval or written directly into a business application. These variants differ in their data flows and permissions.
Explore the system design: Select a component to see its task, incoming and outgoing data, and role in the use case. The guided walkthrough explains each phase. Then use Yes, Uncertain and No to represent your specific system variant.
8 review first11 clarify6 baseline threats
- Present
- Depends on your system
- Absent
- Review first
- To clarify
How to use the model
The diagram shows the typical architecture. Select a component to see what happens there and adapt its features to your system. Enable “Analysis guidance” to see which threats need reviewing for this architecture. The page's checklist continues to describe the typical case.
Changes from the typical case
None yet. Each change will appear here.
Where most threats arise
- 5 clarify
- 3 review first, 1 clarify
- 1 review first, 1 clarify
- 1 review first, 1 clarify
In this system design: 8 to review first, 11 to clarify, 6 baseline threats.
Architecture as a list
External access: Depends on your system
To clarify
Input: Part of every AI system
Review first
To clarify
Baseline for every AI system
Third-party content: Depends on your system
AI model: Part of every AI system
Review first
- Model Drift & Degradation Review priority
Provider service: Present
Review first
To clarify
Training and feedback: Present
Review first
To clarify
Result: Part of every AI system
Review first
- Insecure Output Handling Review priority
To clarify
Business application or equipment: Depends on your system
To clarify
person: Part of every AI system
Review first
- Overreliance Review priority
Affected person: Present
Unapproved AI services: Depends on your system
To clarify
Supply chain and technical foundations: 5 baseline threats applying to every AI system
Baseline for every AI system
How it differs from similar use cases
Clinical decision support (such as early sepsis detection)
Continuously analyses vital signs, laboratory values and patient records rather than individual images
Visual and acoustic quality inspection
Inspects components and products in manufacturing
Data and information
Processing and operating environment
Systems generally arrive as a manufacturer's medical device.
Storage and further use
Some hospitals adapt models using their own images; most use the product unchanged. Corrected findings can return to the manufacturer for future versions.
Results and data sharing
Depending on integration, the system writes results into the radiology information system or reorders the worklist. Cloud image analysis sends health data outside the organisation's environment.
Using a provider does not automatically determine where processing takes place. Storage location, subcontractors, deletion periods and use for training must be checked for the specific deployment.
Assess risks and determine measures
From understanding the system to analysing risks
- Define the system boundary
Record data, participants, interfaces and permissions. Resolve uncertain characteristics in the model.
- Investigate threats
Relate review candidates to actual data flows: what could happen, and who would be affected?
- Assess risk
Assess impacts and likelihood in the deployment context. The number of review candidates is not a risk score.
- Provide evidence of measures
Select appropriate controls, assign responsibilities and define effectiveness checks. Document residual risks and approvals.
Key threats in the application context
- 3 particularly important
- 5 typical
- 11 depending on the system
- 6 additional threats for every AI system
A total of 25 threats from the threat catalogue may be relevant to this use case.
Also review: whether the outputs are substantively correct
Check probability reliability, possible test-data leakage into training and the consequences of an error in the business process.
Also review: whether the system disadvantages groups
Check whether selection, ranking or assessment disadvantages people with protected characteristics, such as sex, age or origin.
Also review: Safety of people and equipment
Check independent safety functions, intended operating limits and the safety evidence your sector requires.
Priority review points
Clinicians reviewing many AI-marked images may increasingly focus on marked regions. Unmarked findings are then more easily missed.
New X-ray equipment, different acquisition settings or a different patient group changes images without informing the model. Detection performance silently falls below the level used for product assessment.
If the system prioritises its worklist by classification, apparently normal images move down the queue. A misclassification delays review of an urgent finding.
5 further review candidates in the typical design
Other relevant threats
Targeted Poisoning / Label Poisoning
Falsified training labels can deliberately shift model outputs.
Listed because of this characteristic
- Learns from examples with correct answers
Training data or an adopted base model can introduce hidden triggers for misbehaviour.
Listed because of this characteristic
- Uses another provider's model
Targeted queries can recover some personal or protected training data from the model.
Listed because of these characteristics
- Processes personal data
- Processes confidential data
Sensitive Information Disclosure
Confidential content can appear in answers, supplied context or logs.
Listed because of these characteristics
- Processes personal data
- Processes confidential data
Exfiltration from ML Application
Protected data can leak through tools, network connections, storage or insiders.
Listed because of this characteristic
- Processes confidential data
Up to 11 further threats may apply, depending on your system design. Nine threats need to be reviewed for every AI system. Three of these are particularly important here and are therefore listed above.
System features and open questions
Each Yes adds further threats to your checklist, up to 11 in total. If you cannot yet answer a question, include its threats in your review.
- Added by this Yes
- Also requires a Yes to another question
How your system is built
- Generates its own text2 threats, plus 1 only in combination with other characteristics
- Processes text, images and speech together2 threats
- Learns from your own data1 threat, plus 1 only in combination with other characteristics
- Changes data or triggers processes1 threat
- Uses operational feedback1 threat
How you operate the system
- Available to people outside the organisation5 threats
- Accepts queries directly through an interface4 threats, plus 1 only in combination with other characteristics
- Accessible over a network2 threats, plus 1 only in combination with other characteristics
- Incurs costs with each query1 threat
- Shares computing resources with other customers or applications1 threat
- Used alongside unapproved AI services1 threat
How your system is built
Does the system generate its own text, such as answers, drafts or summaries?
Most systems mark abnormalities and classify images; some also draft findings.
If yes, add
Can the system change things itself, such as storing data, sending messages, initiating transactions or granting permissions?
Depending on integration, the system writes results into the radiology information system or reorders the worklist.
If yes, add
Does your organisation train or fine-tune the model itself using its own data?
Some hospitals adapt models using their own images; most use the product unchanged.
If yes, add
Does operational feedback or output later feed into analysis or training?
Corrected findings can return to the manufacturer for future versions.
If yes, add
Does the system process several input types together, such as text with images or speech?
Some systems use earlier images or clinical information alongside the current image.
If yes, add
How you operate the system
These questions arise for almost every AI system.
Can people or programs query the model directly, for example through an interface?
If yes, add
Is the system accessible over a network or dependent on network services?
If yes, add
Do you pay per query or unit of computing time, so costs increase with use?
If yes, add
Does the system share servers or computing resources with other customers or applications?
If yes, add
Can people outside your organisation use the system, including without login?
If yes, add
Do staff also use AI services for this task that your organisation has not approved?
If yes, add
Controls and implementation
Measures for the priority review areas are expanded. Each linked measure explains how to implement it.
Review priorityOverreliance3 measures
Review priorityModel Drift & Degradation3 measures
Review priorityInsecure Output Handling3 measures
Targeted Poisoning / Label Poisoning3 measures
Backdoor ML Model2 measures
Privacy Attacks3 measures
Sensitive Information Disclosure3 measures
Exfiltration from ML Application3 measures
Controls for the remaining threats are available on their pages in the threat catalogue.
AI Act provisions
-
Art. 4: AI literacy
For every AI system Applies since 2 February 2025.
-
Art. 6 para. 1, Annex I Section A: Safety component of an assessed product
In typical use Review trigger from 2 August 2028.
Applies to every AI system
Art. 4: AI literacy
Applies if staff or appointed representatives operate or use AI systems.
Applies since 2 February 2025.
Obligations and details for Art. 4: AI literacy
- Who bears the obligations
- Provider, Deployer, Product manufacturer, Provider of a general-purpose AI model
Detailed obligations
May apply in typical use
Art. 6 para. 1, Annex I Section A: Safety component of an assessed product
Applies if the system is marketed as a medical device under Regulation (EU) 2017/745 and requires notified body assessment, usually the case for diagnostic and early warning software under Rule 11.
Applies from 2 August 2028.
Obligations and details for Art. 6 para. 1, Annex I Section A: Safety component of an assessed product
- The general condition is
- Applies if the system is itself a product or a safety component of a product requiring third-party assessment under Annex I legislation, such as machinery, lifts, medical devices or toys.
- Exception
- Systems improving only operation, performance or comfort without a safety role are not safety components under Article 6(1a)–(1c).
- Who bears the obligations
- Provider, Deployer, Product manufacturer
Detailed obligations
Classify your system with the quick checker All obligations in the obligations compass
Cross-cutting threats
Review these threats for every AI system, whatever its purpose.
- Supply Chain: Infrastructure
- Supply Chain: Models
- Supply Chain: Datasets
- Adversarial Inputs
- Insecure Output Handling Review priority: Insecure Output Handling
- Application Vulnerabilities
- Overreliance Review priority: Overreliance
- Model Drift & Degradation Review priority: Model Drift & Degradation
- Middleware Exploits (AI Framework Attacks)
Atypical system features
View 15 further questions
If any of these features does apply to your system, add the listed threats.
Does the system generate or edit images, sounds, voices or videos?
If yes, add
Does the system create new content from open-ended requests that people read, see or hear directly? Translation, reading aloud and transcription do not count.
If yes, add
Can users give the system natural language tasks that it then performs?
The system analyses images and does not accept natural language instructions.
If yes, add
Does the system search its own collection of documents or records for relevant content?
If yes, add
Does the system formulate answers from retrieved documents?
If yes, add
Does the system use a knowledge graph storing concepts and their relationships?
If yes, add
Does the system remember content from earlier conversations or tasks and reuse it later?
If yes, add
Does the system call other programs, interfaces or search services itself, even if only to read?
If yes, add
Does the system determine its next steps itself based on earlier results?
If yes, add
Does the system delegate tasks to other AI agents or incorporate their results?
If yes, add
Is the system connected to tools or data sources through the Model Context Protocol (MCP)?
If yes, add
Do documents from several customers, departments or permission groups share one search store?
If yes, add
Does the system generate code, scripts or database queries for further use?
If yes, add
Does the system execute programs, scripts or commands itself?
If yes, add
Is the system intended to maximise a specified metric, such as revenue, utilisation or clicks?
If yes, add
Sources
- Specialist study
University of Lübeck, AI improves breast cancer detection (PRAIM study), 7 January 2025
The study describes AI in routine German mammography screening, with radiologists still making the findings themselves.
- Register or case collection
Sivakumar et al., FDA Approval of AI/ML Devices in Radiology, JAMA Network Open 2025
The analysis finds that radiology accounts for most AI medical devices, predominantly with 510(k) clearance rather than approval in the narrower sense.
- Register or case collection
U.S. Food and Drug Administration, Artificial Intelligence-Enabled Medical Devices
The official inventory lists AI medical devices permitted for US marketing, mostly for radiology.
- Legislation, norm or standard
DIN and DKE, German Standardisation Roadmap for Artificial Intelligence, Second edition
The roadmap treats medicine as a distinct field.
Related use cases
Also in the field of Healthcare and emergency services
- Clinical decision support (such as early sepsis detection)The system continuously analyses vital signs, laboratory values and patient records and alerts the care team to a possible deterioration.
- AI support for emergency call handlingThe system listens during an emergency call and alerts dispatchers to signs of an emergency, such as cardiac arrest.
Often used together with
Updated: 30 September 2026. Review checklist rules version 0.3.2, threat catalogue version 2026.10.24.2. How these pages are structured