Input data and information sources
Candidates write or speak their answers themselves. Each answer links to a candidate's name and examination number.
Automated assessment of exam performance (essay scoring)
Also known as: Automated scoring of exams and essays
A model assesses written or spoken exam answers and awards marks, alone or alongside human examiners.
Understand the use case
What defines this use case?
The model learns from answers already marked by people and estimates a new answer's score. Major international testing providers state that they use it for admissions and language tests.
This page describes a typical system design. Actual data flows and permissions depend on your implementation.
- System design
- Analysis and processing
- Ability to act
- Action permissions depend on integration
- Fields of use
- education, language examinations
- Also known as
- automated essay assessment, automated language test scoring
How it works, step by step
System design and data flows
Input data
Written answers or voice recordings arrive with the task.
Candidates write or speak their answers themselves. Jointly assessing speaking and writing processes audio and text.
Processing
The model measures structure, vocabulary, grammar or pronunciation and estimates a score.
The model evaluates aspects of submitted work against a defined marking scheme or learned comparison patterns. This produces component assessments, feedback or a suggested grade. What matters is which aspects of performance the method actually captures and how they are combined into an assessment.
Results
The model awards a score for each answer or exam section.
The result describes the defined model task. A match, category or forecast acquires its professional significance only in conjunction with the underlying data and intended use.
Use and impact
The score contributes alone or with human marks to the result and helps determine admission or certification.
Integration determines whether the result is simply displayed, adopted after approval or written directly into a business application. These variants differ in their data flows and permissions.
Explore the system design: Select a component to see its task, incoming and outgoing data, and role in the use case. The guided walkthrough explains each phase. Then use Yes, Uncertain and No to represent your specific system variant.
7 review first15 clarify7 baseline threats
- Present
- Depends on your system
- Absent
- Review first
- To clarify
How to use the model
The diagram shows the typical architecture. Select a component to see what happens there and adapt its features to your system. Enable “Analysis guidance” to see which threats need reviewing for this architecture. The page's checklist continues to describe the typical case.
Changes from the typical case
None yet. Each change will appear here.
Where most threats arise
- 5 clarify
- 3 review first, 1 clarify
- 4 clarify
- 3 review first
In this system design: 7 to review first, 15 to clarify, 7 baseline threats.
Architecture as a list
External access: Depends on your system
To clarify
Input: Part of every AI system
Review first
To clarify
Third-party content: Present
AI model: Part of every AI system
To clarify
- Sleepy Agent (Time/Event-Triggered Hidden Instructions)
- Prompt Injection: Direct
- Prompt Injection: Indirect
- Jailbreaks
- Meta Prompt Extraction
Baseline for every AI system
Provider service: Depends on your system
To clarify
Training and feedback: Present
Review first
Result: Part of every AI system
Baseline for every AI system
Business application or equipment: Depends on your system
To clarify
person: Part of every AI system
Review first
- Overreliance Review priority
Affected person: Present
Unapproved AI services: Depends on your system
To clarify
Supply chain and technical foundations: 5 baseline threats applying to every AI system
Baseline for every AI system
How it differs from similar use cases
Monitors candidates' behaviour during an examination
Data and information
Processing and operating environment
Universities or schools using a ready-made scoring model rely on another provider.
Storage and further use
Testing providers train models on their own previously marked answers. New human assessments can train the next version.
Results and data sharing
PTE Academic uses automated scores without human examiners; e-rater supplements human assessments.
Using a provider does not automatically determine where processing takes place. Storage location, subcontractors, deletion periods and use for training must be checked for the specific deployment.
Assess risks and determine measures
From understanding the system to analysing risks
- Define the system boundary
Record data, participants, interfaces and permissions. Resolve uncertain characteristics in the model.
- Investigate threats
Relate review candidates to actual data flows: what could happen, and who would be affected?
- Assess risk
Assess impacts and likelihood in the deployment context. The number of review candidates is not a risk score.
- Provide evidence of measures
Select appropriate controls, assign responsibilities and define effectiveness checks. Document residual risks and approvals.
Key threats in the application context
- 3 particularly important
- 4 typical
- 15 depending on the system
- 7 additional threats for every AI system
A total of 29 threats from the threat catalogue may be relevant to this use case.
Also review: whether the system disadvantages groups
Check whether selection, ranking or assessment disadvantages people with protected characteristics, such as sex, age or origin.
Priority review points
Candidates learn which features raise scores, such as length, rare words or sentence patterns. The system then awards high marks to weak content.
Targeted Poisoning / Label Poisoning
The model adopts its human markers' standards. If they scored candidates with a certain accent more harshly, the system continues that pattern.
A second examiner who sees the automated score first may be influenced by it. Human assessment then ceases to check it independently.
4 further review candidates in the typical design
Other relevant threats
Your own training or operational feedback can introduce manipulated data into training datasets.
Listed because of this characteristic
- Learns from your own data
Training data or an adopted base model can introduce hidden triggers for misbehaviour.
Listed because of this characteristic
- Learns from your own data
Targeted queries can recover some personal or protected training data from the model.
Listed because of this characteristic
- Processes personal data
Sensitive Information Disclosure
Confidential content can appear in answers, supplied context or logs.
Listed because of this characteristic
- Processes personal data
Up to 15 further threats may apply, depending on your system design. Nine threats need to be reviewed for every AI system. Two of these are particularly important here and are therefore listed above.
System features and open questions
Each Yes adds further threats to your checklist, up to 15 in total. If you cannot yet answer a question, include its threats in your review.
- Added by this Yes
How your system is built
- Performs natural language tasks5 threats
- Processes text, images and speech together2 threats
- Changes data or triggers processes1 threat
- Processes confidential data1 threat
How you operate the system
- Accepts queries directly through an interface4 threats
- Accessible over a network3 threats
- Incurs costs with each query1 threat
- Passes content to services outside your environment1 threat
- Shares computing resources with other customers or applications1 threat
- Used alongside unapproved AI services1 threat
How your system is built
Can users give the system natural language tasks that it then performs?
A language model assessing answers also follows instructions a candidate inserts into an answer.
If yes, add
Can the system change things itself, such as storing data, sending messages, initiating transactions or granting permissions?
PTE Academic uses automated scores without human examiners; e-rater supplements human assessments.
If yes, add
Does the system process confidential information such as trade secrets, health or financial data?
Results are confidential; spoken answers contain candidates' voices.
If yes, add
Does the system process several input types together, such as text with images or speech?
Jointly assessing speaking and writing processes audio and text.
If yes, add
How you operate the system
These questions arise for almost every AI system.
Can people or programs query the model directly, for example through an interface?
If yes, add
Is the system accessible over a network or dependent on network services?
If yes, add
Do you pay per query or unit of computing time, so costs increase with use?
If yes, add
Can content leave your environment, for example to a cloud provider, web search or email?
If yes, add
Does the system share servers or computing resources with other customers or applications?
If yes, add
Do staff also use AI services for this task that your organisation has not approved?
Teachers using a freely available chat service to mark essays do so without approval.
If yes, add
Controls and implementation
Measures for the priority review areas are expanded. Each linked measure explains how to implement it.
Review priorityAdversarial Inputs2 measures
Review priorityTargeted Poisoning / Label Poisoning3 measures
Review priorityOverreliance3 measures
Training Data Poisoning2 measures
Backdoor ML Model2 measures
Privacy Attacks3 measures
Sensitive Information Disclosure3 measures
Controls for the remaining threats are available on their pages in the threat catalogue.
AI Act provisions
-
Art. 4: AI literacy
For every AI system Applies since 2 February 2025.
-
Art. 53 para. 1 point b, Annex XII: Embedded general-purpose model
Depending on use Review trigger since 2 August 2025.
-
Annex III no. 3: Education and vocational training
In typical use Review trigger from 2 December 2027.
Applies to every AI system
Art. 4: AI literacy
Applies if staff or appointed representatives operate or use AI systems.
Applies since 2 February 2025.
Obligations and details for Art. 4: AI literacy
- Who bears the obligations
- Provider, Deployer, Product manufacturer, Provider of a general-purpose AI model
Detailed obligations
May apply in typical use
Annex III no. 3: Education and vocational training
Applies if the system evaluates learning outcomes, such as essays or speaking in exams determining admission or graduation.
Applies from 2 December 2027.
Obligations and details for Annex III no. 3: Education and vocational training
- The general condition is
- Applies if the system determines access or admission, evaluates learning outcomes, assesses educational level or supervises examinations.
- Who bears the obligations
- Provider, Deployer, Product manufacturer
Detailed obligations
May apply in particular uses
Art. 53 para. 1 point b, Annex XII: Embedded general-purpose model
Applies if a large language model or another third-party general-purpose model assesses answers.
Applies since 2 August 2025.
Obligations and details for Art. 53 para. 1 point b, Annex XII: Embedded general-purpose model
- The general condition is
- Applies if another provider's general-purpose AI model is incorporated, such as a large language model.
- Points to consider
- The model provider bears its obligations. It must give you the Annex XII information; use it for your own documentation.
- Who bears the obligations
- Provider of a general-purpose AI model
Detailed obligations
Classify your system with the quick checker All obligations in the obligations compass
Cross-cutting threats
Review these threats for every AI system, whatever its purpose.
Atypical system features
View 16 further questions
If any of these features does apply to your system, add the listed threats.
Does the system generate its own text, such as answers, drafts or summaries?
The system awards marks and does not write feedback.
If yes, add
Does the system generate or edit images, sounds, voices or videos?
If yes, add
Does the system create new content from open-ended requests that people read, see or hear directly? Translation, reading aloud and transcription do not count.
If yes, add
Does the system search its own collection of documents or records for relevant content?
If yes, add
Does the system formulate answers from retrieved documents?
If yes, add
Does the system use a knowledge graph storing concepts and their relationships?
If yes, add
Does the system remember content from earlier conversations or tasks and reuse it later?
If yes, add
Does the system call other programs, interfaces or search services itself, even if only to read?
If yes, add
Does the system determine its next steps itself based on earlier results?
If yes, add
Does the system delegate tasks to other AI agents or incorporate their results?
If yes, add
Is the system connected to tools or data sources through the Model Context Protocol (MCP)?
If yes, add
Do documents from several customers, departments or permission groups share one search store?
If yes, add
Can people outside your organisation use the system, including without login?
The system assesses only registered candidates and is not publicly accessible.
If yes, add
Does the system generate code, scripts or database queries for further use?
If yes, add
Does the system execute programs, scripts or commands itself?
If yes, add
Is the system intended to maximise a specified metric, such as revenue, utilisation or clicks?
If yes, add
Sources
- Report of a deployment
ETS, About the e-rater Scoring Engine According to the deployer
ETS describes e-rater working alongside human assessment in high-stakes GRE and TOEFL iBT examinations.
- Report of a deployment
Pearson, PTE scoring explained According to the deployer
Pearson describes automated scoring for every PTE examination.
- Legislation, norm or standard
EU AI Act, Annex III, point 3(b)
The Act expressly lists AI systems evaluating learning outcomes in Annex III.
Related use cases
Also in the field of Employment and education
- AI-assisted candidate screeningThe system uses CVs and cover letters to identify the best match for a role and suggest whom the company should invite for interview.
- Algorithmic managementSoftware assigns tasks and shifts to workers, tracks what they do and evaluates their performance.
- Online proctoringThe system monitors online exam candidates through camera, microphone and screen and flags possible cheating.
Often used together with
Updated: 30 September 2026. Review checklist rules version 0.3.2, threat catalogue version 2026.10.24.2. How these pages are structured