AI threat catalogueAttacks on the Running Model and ServiceProduction
Model Theft
Attackers create a working copy of a proprietary AI model, either by querying it at scale or by breaking into the infrastructure. They steal the intellectual property without ever touching the original.
Description
A proprietary model embodies expensive development work. Attackers reproduce it in two ways. In the first, they query the model at scale through its normal interface and use the collected input-output pairs to train their own imitation, a so-called shadow or clone model that behaves almost identically. In the second, they break into the infrastructure and copy the model files directly. Side channels such as the hardware's electromagnetic emissions can also give details away. Motives include saving usage fees, building a competing product, circumventing export controls, or preparing further attacks.
Possible impact
The organisation loses intellectual property that often forms the basis of its business: a competitor gains comparable capabilities without bearing the development costs. The copied model also serves as a springboard for further attacks, for example to prepare adversarial inputs at leisure. The EU AI Act explicitly names model theft as a threat to be defended against and requires adequate protection of model weights and infrastructure for models with systemic risk.
Example
A provider offers a specialised classification model as a paid interface. A competitor sends millions of automated queries over several weeks, stores the answers and uses them to train its own model that replicates the service at a fraction of the cost.
Recommended mitigations (5)
Every mitigation states its control type, effect, implementation level and the reason for the classification.
Rate limiting and query quotasTechnical
- Effect
- Preventive, Impact-limiting
- Implementation level
- Application, API & agents, Infrastructure
- Complementary control type
- Organizational & process-based
- Reason for the classification
- “Rate limiting and query quotas” is primarily technical: Minimization, randomization, uniform processing, or query limits reduce information gained from repeated observations; complemented by binding workflows.
WatermarkingTechnical
- Effect
- Detective
- Implementation level
- Model & training, Application, API & agents
- Reason for the classification
- “Watermarking” is primarily technical: Cryptographic or machine-verifiable properties protect confidentiality, integrity, or provenance.
Model output perturbationTechnical
- Effect
- Preventive
- Implementation level
- Model & training, Application, API & agents
- Reason for the classification
- “Model output perturbation” is primarily technical: Minimization, randomization, uniform processing, or query limits reduce information gained from repeated observations.
Access control and authenticationTechnical
- Effect
- Preventive
- Implementation level
- Application, API & agents, Infrastructure
- Complementary control type
- Organizational & process-based
- Reason for the classification
- “Access control and authentication” is primarily technical: Machine-enforced identity, permission, or scope rules constrain unauthorized access and actions; complemented by binding workflows.
Monitor for extraction patternsTechnical
- Effect
- Detective
- Implementation level
- Application, API & agents, Use & operations
- Complementary control type
- Organizational & process-based
- Reason for the classification
- “Monitor for extraction patterns” is primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators; complemented by binding workflows.
Framework mappings
Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.
Verified references (8)
Every reference states the framework, the exact location and the publishing organisation.
- OWASP LLM Top 10 LLM10:2025 Unbounded ConsumptionLLM10:2025 Unbounded Consumption, official category page OWASP FoundationOriginal
- NIST AI RMF Section 2.10 Intellectual PropertySection 2.10, p. 11 National Institute of Standards and Technology (NIST)Original
- NIST AI RMF NISTAML.031 Model ExtractionTaxonomy Index, p. x; Section 2.4.4, pp. 31–32; Section 3.3.2, p. 47 National Institute of Standards and Technology (NIST)Original
- MITRE ATLAS AML.T0040 AI Model Inference API AccessATLAS.yaml technique object with id AML.T0040 (pinned release v5.6.0) MITREOriginal
- MITRE ATLAS AML.T0044 Full AI Model AccessATLAS.yaml technique object with id AML.T0044 (pinned release v5.6.0) MITREOriginal
- EU AI Act Article 55(1)(d) Obligations of providers of general-purpose AI models with systemic riskArticle 55(1)(d) European Union (EUR-Lex)Original
- BSI R24 Modelldiebstahl (Text, Bild, Video)Kap. 4, R24, p. 29 Bundesamt für Sicherheit in der Informationstechnik (BSI)Original
- BIML BIML78 model:5 Steal the BoxPDF p. 20, [model:5:steal the box] Berryville Institute of Machine Learning (BIML)Original
Related threats
More entries from the topic group Attacks on the Running Model and Service.
Assess this threat in your own system
The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.
Cite this entry
For reports, policies or internal documents; the link leads directly to this entry.
“Model Theft”. Versatile AI Risk Assessment, AI threat catalogue, as of July 2026. https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/model-theft/