Versatile AI Risk Assessment

AI threat catalogue Model and Training Data Manipulation Development

Targeted Poisoning / Label Poisoning

Attackers tamper with a small, targeted portion of the training data, for example through wrong labels. Overall the model looks fine, but for specific cases chosen by the attacker it reliably makes the wrong call.

As of: October 2026 · Catalogue version 2026.10.24.2 · 4 mitigations · 11 verified sources

Description

In supervised learning, every training example carries a label such as 'fraud' or 'no fraud'. In targeted poisoning, the attacker rewrites these labels for a small subset or adds an inconspicuous trigger pattern to selected examples. The model then learns a hidden exception rule that only fires on the inputs the attacker cares about. Because overall accuracy stays normal, standard quality tests usually do not reveal the manipulation. Every place where data is labeled is a potential entry point: in-house teams, crowdsourcing, or external service providers. Even a small number of manipulated records can be enough.

Possible impact

The organization relies on a system that fails precisely in the cases that matter to the attacker, for example in fraud screening or access decisions. Such blind spots often go unnoticed for a long time and cause financial losses and liability questions. Reports and analyses also lose credibility because they build on corrupted decisions.

Example

An insurer has claim reports labeled by an external provider to train its fraud detection. An insider there consistently marks one particular type of fabricated claim as genuine. The finished model later waves this exact fraud pattern through automatically.

Recommended mitigations (4)

Compare effects and implementation levels. Expand each entry for the additional classification and rationale.

  • Label consistency verification

    Technical
    Effect
    Preventive, Detective
    Implementation level
    Data, Model & training
    Classification and rationale

    Complementary control typeOrganizational & process-based

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators; complemented by binding workflows.

  • Outlier detection in training samples

    Technical
    Effect
    Preventive, Detective
    Implementation level
    Data, Model & training
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

  • Cross-validation with clean holdout sets

    Technical
    Effect
    Detective
    Implementation level
    Data, Model & training
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

  • Data source diversity

    Organizational & process-based
    Effect
    Preventive
    Implementation level
    Data, Organization
    Classification and rationale

    Complementary control typeTechnical

    Primarily organizational and process-based: Defined selection, operating, or lifecycle procedures make the control binding and repeatable; complemented by technical implementation.

Framework mappings

Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.

OWASP LLM Top 10 LLM05:2026NIST AI RMF Section 2.9 · NISTAML.012 · NISTAML.024MITRE ATLAS AML.T0020EU AI Act Article 53(1)(d) · Article 55(1)(a)BSI R17BIML BIML-LLM LLMtop10:6 · BIML78 assembly:2 · BIML78 data:6

Verified references (11)

Every reference states the framework, the exact location and the publishing organisation.

EU AI Act obligations related to this threat

The obligations from the obligations compass whose articles the catalogue maps to this threat. Each leads to its entry with core, action step and evidence; whether it applies to you depends on role and risk class.

  • Draw up the technical documentation of the model Art. 53 Abs. 1 Buchst. a, Anhang XI · applies from 2 August 2025

    Statutory roles: Provider of a general-purpose AI model. Classes: General-purpose AI model, Model with systemic risk.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Provide information to downstream providers Art. 53 Abs. 1 Buchst. b, Anhang XII · applies from 2 August 2025

    Statutory roles: Provider of a general-purpose AI model. Classes: General-purpose AI model, Model with systemic risk.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Respect copyright and summarise the training content Art. 53 Abs. 1 Buchst. c und d · applies from 2 August 2025

    Statutory roles: Provider of a general-purpose AI model. Classes: General-purpose AI model, Model with systemic risk.

    For models released under a free and open-source licence with publicly available parameters, the obligations under points (a) and (b) do not apply, but the copyright policy and the summary do. Models with systemic risk are excluded from this relief.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Evaluate and mitigate systemic risks Art. 55 · applies from 2 August 2025

    Statutory roles: Provider of a general-purpose AI model. Classes: Model with systemic risk.

    This reference is a review prompt. A threat match alone does not establish applicability.

All 62 obligations by role, risk class and deadline in the obligations compass

More entries from the topic group Model and Training Data Manipulation.

Training Data Poisoning Attackers inject manipulated or false data into the data an AI model learns from. Backdoor ML Model The model contains hidden behavior, a backdoor. It works correctly on normal inputs; only a secret trigger pattern in the input flips the … Sleepy Agent (Time/Event-Triggered Hidden Instructions) Malicious logic lies dormant inside the model and only activates later: on a set date, at a specific event, or in a particular environment.

Matching use cases

This list shows where to review this threat, rather than how high the risk is.

Particularly important for:

Typical for:

For 18 further use cases, relevance depends on the system design, for example whether it uses operational feedback or learns from examples with correct answers. Show these use cases

Frequently asked questions

What is Targeted Poisoning / Label Poisoning?
Attackers tamper with a small, targeted portion of the training data, for example through wrong labels. Overall the model looks fine, but for specific cases chosen by the attacker it reliably makes the wrong call.
Which mitigations help against Targeted Poisoning / Label Poisoning?
The catalogue lists 4 mitigations, each with control type, effect and implementation level. First among them are Label consistency verification, Outlier detection in training samples and Cross-validation with clean holdout sets.
Which EU AI Act obligations does Targeted Poisoning / Label Poisoning touch?
The catalogue maps this threat to Art. 53 Abs. 1 Buchst. a, Art. 53 Abs. 1 Buchst. b, Art. 53 Abs. 1 Buchst. c und d and Art. 55 of the EU AI Act. In the obligations compass this leads to: Draw up the technical documentation of the model, Provide information to downstream providers, Respect copyright and summarise the training content and Evaluate and mitigate systemic risks. Whether an obligation applies depends on role, risk class and deadline.

Assess this threat in your own system

The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.

Start the live demo See the free modules

Cite this entry

For reports, policies or internal documents; the link leads directly to this entry.

“Targeted Poisoning / Label Poisoning”. Versatile AI Risk Assessment, AI threat catalogue, as of October 2026.
https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/targeted-label-poisoning/

← Back to the full catalogue