Input data and information sources
The system analyses images and voices of people who control their own behaviour and can disguise it. The system processes identifiable people's faces or voices.
Emotion recognition
The system infers a person's emotions or intentions from their face, voice or behaviour.
Understand the use case
What defines this use case?
Trials have taken place at UK railway stations and in an EU research project at border crossings.
This page describes a typical system design. Actual data flows and permissions depend on your implementation.
- System design
- Analysis and processing
- Ability to act
- Action permissions depend on integration
- Fields of use
- transport operators, border control
- Also known as
- emotion analysis from faces or voices
How it works, step by step
System design and data flows
Input data
The system receives camera images, voice recordings or other bodily signals from a person.
The system analyses images and voices of people who control their own behaviour and can disguise it. Some systems analyse face and voice together; others use only one.
Processing
The model recognises expressions, tone or movement and infers an emotion such as joy, anger or fear, or an intention such as deception.
The model processes characteristics of the face, voice or other signals and assigns an emotion category or assessment. It thus infers a state that cannot be observed directly from external signals. The resulting category is a model interpretation rather than a direct measurement of subjective experience.
Results
The system assigns an emotion and numerical score to each person or group.
The result describes the defined model task. A match, category or forecast acquires its professional significance only in conjunction with the underlying data and intended use.
Use and impact
The output feeds into analysis or influences how workers, customers or travellers are treated.
Integration determines whether the result is simply displayed, adopted after approval or written directly into a business application. These variants differ in their data flows and permissions.
Explore the system design: Select a component to see its task, incoming and outgoing data, and role in the use case. The guided walkthrough explains each phase. Then use Yes, Uncertain and No to represent your specific system variant.
7 review first10 clarify7 baseline threats
- Present
- Depends on your system
- Absent
- Review first
- To clarify
How to use the model
The diagram shows the typical architecture. Select a component to see what happens there and adapt its features to your system. Enable “Analysis guidance” to see which threats need reviewing for this architecture. The page's checklist continues to describe the typical case.
Changes from the typical case
None yet. Each change will appear here.
Where most threats arise
- 4 review first, 1 clarify
- 4 clarify
- 1 review first, 2 clarify
- 1 review first, 1 clarify
In this system design: 7 to review first, 10 to clarify, 7 baseline threats.
Architecture as a list
External access: Depends on your system
To clarify
Input: Part of every AI system
Review first
- Adversarial Inputs Review priority
- Sensitive Information Disclosure Review priority
- Privacy Attacks
- Exfiltration from ML Application
To clarify
Third-party content: Present
AI model: Part of every AI system
Baseline for every AI system
Provider service: Present
Review first
To clarify
Training and feedback: Present
Review first
To clarify
Result: Part of every AI system
Baseline for every AI system
Business application or equipment: Depends on your system
To clarify
person: Part of every AI system
Review first
- Overreliance Review priority
Affected person: Depends on your system
Unapproved AI services: Depends on your system
To clarify
Supply chain and technical foundations: 5 baseline threats applying to every AI system
Baseline for every AI system
How it differs from similar use cases
Text analytics for feedback and consultation responses
Determines sentiment in written text
Remote biometric identification (facial recognition)
Identifies a person from their face rather than determining their emotions
Data and information
Processing and operating environment
Deployers use ready-made services, such as Amazon Rekognition at Network Rail.
Storage and further use
Logging, retention periods and possible use for training must be established for the specific implementation.
Results and data sharing
If a cloud service analyses the images, they leave the organisation's environment.
Using a provider does not automatically determine where processing takes place. Storage location, subcontractors, deletion periods and use for training must be checked for the specific deployment.
Assess risks and determine measures
From understanding the system to analysing risks
- Define the system boundary
Record data, participants, interfaces and permissions. Resolve uncertain characteristics in the model.
- Investigate threats
Relate review candidates to actual data flows: what could happen, and who would be affected?
- Assess risk
Assess impacts and likelihood in the deployment context. The number of review candidates is not a risk score.
- Provide evidence of measures
Select appropriate controls, assign responsibilities and define effectiveness checks. Document residual risks and approvals.
Key threats in the application context
- 3 particularly important
- 4 typical
- 10 depending on the system
- 7 additional threats for every AI system
A total of 24 threats from the threat catalogue may be relevant to this use case.
Priority review points
Facial expressions do not reliably reveal feelings and differ across cultures and individuals. Trusting the output may label a tense traveller suspicious or a calm worker unmotivated.
People can deliberately control their expressions and tone when they know they are observed. The system then attributes calmness or honesty where neither exists.
Sensitive Information Disclosure
The system records its attributed emotion for each recording. If these labels leak with an image or name, individuals' supposed moods, fears or intentions become known.
4 further review candidates in the typical design
Other relevant threats
Targeted Poisoning / Label Poisoning
Falsified training labels can deliberately shift model outputs.
Listed because of this characteristic
- Learns from examples with correct answers
Training data or an adopted base model can introduce hidden triggers for misbehaviour.
Listed because of this characteristic
- Uses another provider's model
Targeted queries can recover some personal or protected training data from the model.
Listed because of these characteristics
- Processes personal data
- Processes confidential data
- Recognises or compares people by physical features
Exfiltration from ML Application
Protected data can leak through tools, network connections, storage or insiders.
Listed because of this characteristic
- Processes confidential data
Up to 10 further threats may apply, depending on your system design. Nine threats need to be reviewed for every AI system. Two of these are particularly important here and are therefore listed above.
System features and open questions
Each Yes adds further threats to your checklist, up to 10 in total. If you cannot yet answer a question, include its threats in your review.
- Added by this Yes
- Also requires a Yes to another question
How your system is built
- Processes text, images and speech together2 threats
- Learns from your own data1 threat, plus 1 only in combination with other characteristics
- Changes data or triggers processes1 threat
- Uses operational feedback1 threat
How you operate the system
- Available to people outside the organisation5 threats
- Accepts queries directly through an interface4 threats
- Accessible over a network2 threats, plus 1 only in combination with other characteristics
- Incurs costs with each query1 threat
- Shares computing resources with other customers or applications1 threat
- Used alongside unapproved AI services1 threat
How your system is built
Can the system change things itself, such as storing data, sending messages, initiating transactions or granting permissions?
If yes, add
Does your organisation train or fine-tune the model itself using its own data?
If yes, add
Does operational feedback or output later feed into analysis or training?
If yes, add
Does the system process several input types together, such as text with images or speech?
Some systems analyse face and voice together; others use only one.
If yes, add
How you operate the system
These questions arise for almost every AI system.
Can people or programs query the model directly, for example through an interface?
If yes, add
Is the system accessible over a network or dependent on network services?
With a cloud service, camera images travel over the network for analysis.
If yes, add
Do you pay per query or unit of computing time, so costs increase with use?
If yes, add
Does the system share servers or computing resources with other customers or applications?
If yes, add
Can people outside your organisation use the system, including without login?
If yes, add
Do staff also use AI services for this task that your organisation has not approved?
If yes, add
Controls and implementation
Measures for the priority review areas are expanded. Each linked measure explains how to implement it.
Review priorityOverreliance3 measures
Review priorityAdversarial Inputs2 measures
Review prioritySensitive Information Disclosure3 measures
Targeted Poisoning / Label Poisoning3 measures
Backdoor ML Model2 measures
Privacy Attacks3 measures
Exfiltration from ML Application3 measures
Controls for the remaining threats are available on their pages in the threat catalogue.
AI Act provisions
-
Art. 4: AI literacy
For every AI system Applies since 2 February 2025.
-
Art. 5 para. 1 point e, f, g, h: Prohibited biometric practices
In typical use Review trigger since 2 February 2025.
-
Art. 50 para. 3: Disclosure of emotion recognition and categorisation
Depending on use Review trigger since 2 August 2026.
-
Annex III no. 1 point c: Emotion recognition
In typical use Review trigger from 2 December 2027.
-
Annex III no. 7: Migration, asylum and border control
Depending on use Review trigger from 2 December 2027.
Applies to every AI system
Art. 4: AI literacy
Applies if staff or appointed representatives operate or use AI systems.
Applies since 2 February 2025.
Obligations and details for Art. 4: AI literacy
- Who bears the obligations
- Provider, Deployer, Product manufacturer, Provider of a general-purpose AI model
Detailed obligations
May apply in typical use
Art. 5 para. 1 point e, f, g, h: Prohibited biometric practices
Applies if the system recognises workers' emotions in the workplace or learners' emotions in an educational institution, except where used for medical or safety reasons.
Applies since 2 February 2025.
Obligations and details for Art. 5 para. 1 point e, f, g, h: Prohibited biometric practices
- The general condition is
- Applies if the system indiscriminately collects facial images from the internet or video, recognises workplace or educational emotions, categorises people biometrically by sensitive attributes, or identifies people in public spaces in real time for law enforcement.
- Exception
- Workplace and educational emotion recognition is permitted for medical or safety reasons. Real-time law enforcement identification has narrowly defined exceptions.
- Who bears the obligations
- Provider, Deployer, Importer, Distributor, Authorised representative, Product manufacturer, Provider of a general-purpose AI model
Detailed obligations
Annex III no. 1 point c: Emotion recognition
Applies if the system infers emotions or intentions from the faces, voices or bodily signals of people other than workers or learners, such as travellers or telephone customers.
Applies from 2 December 2027.
Obligations and details for Annex III no. 1 point c: Emotion recognition
- The general condition is
- Applies if the system infers emotions or intentions from faces, voices or bodily signals.
- Who bears the obligations
- Provider, Deployer, Product manufacturer
Detailed obligations
May apply in particular uses
Annex III no. 7: Migration, asylum and border control
Applies if authorities use the system at borders or in asylum procedures to detect deception, as trialled by the iBorderCtrl research project.
Applies from 2 December 2027.
Obligations and details for Annex III no. 7: Migration, asylum and border control
- The general condition is
- Applies if the system is used in examining asylum, visa or residence applications or at border controls.
- Who bears the obligations
- Provider, Deployer, Product manufacturer
Detailed obligations
Art. 50 para. 3: Disclosure of emotion recognition and categorisation
Applies if people are exposed to the system, for example at a station or during a customer conversation.
Applies since 2 August 2026.
Obligations and details for Art. 50 para. 3: Disclosure of emotion recognition and categorisation
- The general condition is
- Applies if the system recognises emotions or categorises people biometrically.
- Who bears the obligations
- Deployer
Detailed obligations
Classify your system with the quick checker All obligations in the obligations compass
Cross-cutting threats
Review these threats for every AI system, whatever its purpose.
Atypical system features
View 16 further questions
If any of these features does apply to your system, add the listed threats.
Does the system generate its own text, such as answers, drafts or summaries?
The system produces an emotion label or score for each person, rather than text.
If yes, add
Does the system generate or edit images, sounds, voices or videos?
If yes, add
Does the system create new content from open-ended requests that people read, see or hear directly? Translation, reading aloud and transcription do not count.
If yes, add
Can users give the system natural language tasks that it then performs?
If yes, add
Does the system search its own collection of documents or records for relevant content?
If yes, add
Does the system formulate answers from retrieved documents?
If yes, add
Does the system use a knowledge graph storing concepts and their relationships?
If yes, add
Does the system remember content from earlier conversations or tasks and reuse it later?
If yes, add
Does the system call other programs, interfaces or search services itself, even if only to read?
If yes, add
Does the system determine its next steps itself based on earlier results?
If yes, add
Does the system delegate tasks to other AI agents or incorporate their results?
If yes, add
Is the system connected to tools or data sources through the Model Context Protocol (MCP)?
If yes, add
Do documents from several customers, departments or permission groups share one search store?
If yes, add
Does the system generate code, scripts or database queries for further use?
If yes, add
Does the system execute programs, scripts or commands itself?
If yes, add
Is the system intended to maximise a specified metric, such as revenue, utilisation or clicks?
If yes, add
Sources
- Authority, court or parliament
European Commission, Guidelines on prohibited AI practices, 2025
The Commission explains that recognising call centre workers' emotions is prohibited, whereas analysing customers' emotions is not.
- Legislation, norm or standard
EU AI Act, Annex III, point 1(c)
The Act expressly lists emotion recognition systems in Annex III where they are not prohibited.
- Authority, court or parliament
Federal Network Agency, Prohibited practices
The agency gives the example of a webcam used to detect workers' anger.
- Report of a deployment
DIGIT, Hidden AI Cameras Recorded Train Passengers’ Emotions at Glasgow Central, 19 June 2024
The report describes an emotion analysis trial at several UK railway stations.
- Specialist study
CORDIS, iBorderCtrl (Grant 700626), Reporting page
The page describes deception detection as part of a border control system and trials in three countries.
Related use cases
Also in the field of Biometrics and emotion recognition
- Remote biometric identification (facial recognition)The system compares faces in camera images or photographs with a database to identify unknown people.
- Biometric verification and video identificationThe system checks whether a person is who they claim to be, for example by comparing a live image with their ID photograph.
Often used together with
- Intelligent video analytics in public spacesThe system detects specific events or objects in camera images and alerts a control centre without identifying people.
- Customer service chatbots and voicebotsA bot independently handles customers' or citizens' requests in chat or by phone and passes difficult cases to people.
Updated: 30 September 2026. Review checklist rules version 0.3.2, threat catalogue version 2026.10.24.2. How these pages are structured