AI threat catalogue Supply Chain and Provenance Supply Chain
Supply Chain – Datasets
Training and fine-tuning data from external sources can be poisoned, flawed, or legally tainted. The model learns these defects along with everything else; beyond skewed or harmful outputs, copyright and data protection violations can follow.
Description
AI models learn from large volumes of data that are often gathered automatically from the internet (crawling) or bought in from third parties, frequently without deeper integrity checks. Attackers exploit this: they place prepared content in sources that feed into training data, or take over expired internet domains listed in well-known dataset catalogs and replace their content. Poisoned data thus enters training or fine-tuning (the subsequent adjustment of a finished model) and embeds bias, false information, or hidden backdoors. External datasets also carry legal risks, such as copyrighted material or personal data collected unlawfully. If data provenance is not documented, the defect often goes undetected for a long time.
Possible impact
Poisoned or defective data lowers the quality and reliability of the model and can implant deliberately harmful behavior. The organization faces copyright disputes and GDPR violations if personal data flows in without a legal basis; the individuals whose data is processed unnoticed are affected too. Depending on role and risk class, the EU AI Act requires safeguards against data poisoning and transparency about training data. Clean-up and retraining costs and reputational damage come on top.
Example
A company buys an industry dataset to fine-tune its model for credit decisions. Part of the data comes from manipulated web sources and contains systematically skewed examples. The model then disadvantages certain customer groups without this showing up in standard testing.
Recommended mitigations (4)
Compare effects and implementation levels. Expand each entry for the additional classification and rationale.
-
Use trusted data sources
Organizational & process-based- Effect
- Preventive
- Implementation level
- Data, Supply chain
Classification and rationale
Complementary control typeTechnical
Primarily organizational and process-based: Defined selection, operating, or lifecycle procedures make the control binding and repeatable; complemented by technical implementation.
-
Data provenance tracking
Technical- Effect
- Detective
- Implementation level
- Data
Classification and rationale
Complementary control typeOrganizational & process-based
Primarily technical: Cryptographic or machine-verifiable properties protect confidentiality, integrity, or provenance; complemented by binding workflows.
-
Dataset validation and integrity checks
Technical- Effect
- Preventive, Detective
- Implementation level
- Data
Classification and rationale
Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.
-
License and copyright compliance audits
Governance & compliance- Effect
- Detective
- Implementation level
- Data, Supply chain, Organization
Classification and rationale
Complementary control typeOrganizational & process-based
Primarily a governance and compliance control: Binding rules, control objectives, or oversight define permitted use and accountability; complemented by binding workflows.
Framework mappings
Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.
Verified references (18)
Every reference states the framework, the exact location and the publishing organisation.
- OWASP LLM Top 10 LLM04:2026 Supply ChainLLM04:2026 Supply Chain, OWASP Top 10 for LLM Applications 2026 v1.0, pp. 27-32 (doi:10.5281/zenodo.22109015); canonical text 2026/final/LLM04_SupplyChain.md at commit 7e144e05 OWASP FoundationOriginal
- OWASP LLM Top 10 LLM05:2026 Data and Model PoisoningLLM05:2026 Data and Model Poisoning, OWASP Top 10 for LLM Applications 2026 v1.0, pp. 33-37 (doi:10.5281/zenodo.22109015); canonical text 2026/final/LLM05_DataModelPoisoning.md at commit 7e144e05 OWASP FoundationOriginal
- NIST AI RMF Section 2.10 Intellectual PropertySection 2.10, p. 11 National Institute of Standards and Technology (NIST)Original
- NIST AI RMF Section 2.12 Value Chain and Component IntegrationSection 2.12, p. 12 National Institute of Standards and Technology (NIST)Original
- NIST AI RMF MAP 4.1 MAP 4.1MAP 4.1, p. 27 National Institute of Standards and Technology (NIST)Original
- NIST AI RMF MEASURE 2.5 MEASURE 2.5MEASURE 2.5, p. 29 National Institute of Standards and Technology (NIST)Original
- NIST AI RMF NISTAML.05 Supply Chain AttacksTaxonomy Index, pp. x–xi; Section 3.2, pp. 41–43 National Institute of Standards and Technology (NIST)Original
- MITRE ATLAS AML.T0010 AI Supply Chain CompromiseATLAS-2026.09.yaml technique object with id AML.T0010 (content release 2026.09, format-version 6.0.0) MITREOriginal
- MITRE ATLAS AML.T0115.000 DatasetsATLAS-2026.09.yaml technique object with id AML.T0115.000 (content release 2026.09, format-version 6.0.0) MITREOriginal
- EU AI Act Article 25(4) Responsibilities along the AI value chainArticle 25(4), first subparagraph, as replaced by Article 1(12)(b) of Regulation (EU) 2026/1744 European Union (EUR-Lex)Original
- EU AI Act Article 53(1)(a) Obligations for providers of general-purpose AI modelsArticle 53(1)(a) and Annex XI European Union (EUR-Lex)Original
- EU AI Act Article 53(1)(d) Obligations for providers of general-purpose AI modelsArticle 53(1)(d) European Union (EUR-Lex)Original
- GDPR Article 25(1)–(2) Data protection by design and by defaultArticle 25(1) and 25(2) European Union (EUR-Lex)Original
- GDPR EDPB Opinion 28/2024, Sections 3.3–3.4.2 EDPB Opinion 28/2024 on certain data protection aspects related to the processing of personal data in the context of AI modelsSections 3.3–3.4 and Section 3.4.2, especially paragraphs 124–132, pp. 32–33 (unlawful development and downstream use) European Data Protection Board (EDPB)Original
- BSI R17 Vergiftung der Trainingsdaten (Data Poisoning) (Text, Bild, Video)Kap. 4, R17, p. 25 Bundesamt für Sicherheit in der Informationstechnik (BSI)Original
- BIML BIML-LLM LLMtop10:2 Data DebtPDF p. 12, [LLMtop10:2:data debt] Berryville Institute of Machine Learning (BIML)Original
- BIML BIML-LLM LLMtop10:6 Poison in the DataPDF p. 13, [LLMtop10:6:poison in the data] Berryville Institute of Machine Learning (BIML)Original
- BIML BIML78 raw:2 TrustworthinessPDF p. 10, [raw:2:trustworthiness] Berryville Institute of Machine Learning (BIML)Original
EU AI Act obligations related to this threat
The obligations from the obligations compass whose articles the catalogue maps to this threat. Each leads to its entry with core, action step and evidence; whether it applies to you depends on role and risk class.
-
Support downstream providers
Art. 25 Abs. 2, Art. 25 Abs. 4 · applies from Annex III: 2 December 2027 · Annex I: 2 August 2028 · amended by Reg. 2026/1744
Statutory roles: Provider, Provider of a general-purpose AI model. Classes: High-risk under Annex III, High-risk as a product or safety component.
Regulation (EU) 2026/1744 spelled out the initial provider duty to cooperate and at the same time created an exception: it does not apply where the initial provider has clearly specified that their system must not be turned into a high-risk system. Infringements of Art. 25(2) and (4) are now expressly subject to fines.
This reference is a review prompt. A threat match alone does not establish applicability.
-
Draw up the technical documentation of the model
Art. 53 Abs. 1 Buchst. a, Anhang XI · applies from 2 August 2025
Statutory roles: Provider of a general-purpose AI model. Classes: General-purpose AI model, Model with systemic risk.
This reference is a review prompt. A threat match alone does not establish applicability.
-
Provide information to downstream providers
Art. 53 Abs. 1 Buchst. b, Anhang XII · applies from 2 August 2025
Statutory roles: Provider of a general-purpose AI model. Classes: General-purpose AI model, Model with systemic risk.
This reference is a review prompt. A threat match alone does not establish applicability.
-
Respect copyright and summarise the training content
Art. 53 Abs. 1 Buchst. c und d · applies from 2 August 2025
Statutory roles: Provider of a general-purpose AI model. Classes: General-purpose AI model, Model with systemic risk.
For models released under a free and open-source licence with publicly available parameters, the obligations under points (a) and (b) do not apply, but the copyright policy and the summary do. Models with systemic risk are excluded from this relief.
This reference is a review prompt. A threat match alone does not establish applicability.
All 62 obligations by role, risk class and deadline in the obligations compass
Terms on this page
Glossary terms that occur in this entry. Every link leads to the full explanation.
- Data Poisoning Manipulated training or reference data steers a model wrong on purpose.
- Fine-tuning An existing model is trained further on your own data.
Related threats
More entries from the topic group Supply Chain and Provenance.
Matching use cases
This list shows where to review this threat, rather than how high the risk is.
Belongs on the checklist for every AI system. What to review for every AI system
Frequently asked questions
- What is Supply Chain – Datasets?
- Training and fine-tuning data from external sources can be poisoned, flawed, or legally tainted. The model learns these defects along with everything else; beyond skewed or harmful outputs, copyright and data protection violations can follow.
- Which mitigations help against Supply Chain – Datasets?
- The catalogue lists 4 mitigations, each with control type, effect and implementation level. First among them are Use trusted data sources, Data provenance tracking and Dataset validation and integrity checks.
- Which EU AI Act obligations does Supply Chain – Datasets touch?
- The catalogue maps this threat to Art. 25 Abs. 2, Art. 53 Abs. 1 Buchst. a, Art. 53 Abs. 1 Buchst. b and Art. 53 Abs. 1 Buchst. c und d of the EU AI Act. In the obligations compass this leads to: Support downstream providers, Draw up the technical documentation of the model, Provide information to downstream providers and Respect copyright and summarise the training content. Whether an obligation applies depends on role, risk class and deadline.
Assess this threat in your own system
The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.
Cite this entry
For reports, policies or internal documents; the link leads directly to this entry.
“Supply Chain – Datasets”. Versatile AI Risk Assessment, AI threat catalogue, as of October 2026. https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/supply-chain-datasets/