Versatile AI Risk Assessment

AI threat catalogue Attacks on the Running Model and Service Production

Model Theft

Attackers create a working copy of a proprietary AI model, either by querying it at scale or by breaking into the infrastructure. They steal the intellectual property without ever touching the original.

As of: October 2026 · Catalogue version 2026.10.24.2 · 5 mitigations · 8 verified sources

Description

A proprietary model embodies expensive development work. Attackers reproduce it in two ways. In the first, they query the model at scale through its normal interface and use the collected input-output pairs to train their own imitation, a so-called shadow or clone model that behaves almost identically. In the second, they break into the infrastructure and copy the model files directly. Side channels such as the hardware's electromagnetic emissions can also give details away. Motives include saving usage fees, building a competing product, circumventing export controls, or preparing further attacks.

Possible impact

The organisation loses intellectual property that often forms the basis of its business: a competitor gains comparable capabilities without bearing the development costs. The copied model also serves as a springboard for further attacks, for example to prepare adversarial inputs at leisure. The EU AI Act explicitly names model theft as a threat to be defended against and requires adequate protection of model weights and infrastructure for models with systemic risk.

Example

A provider offers a specialised classification model as a paid interface. A competitor sends millions of automated queries over several weeks, stores the answers and uses them to train its own model that replicates the service at a fraction of the cost.

Recommended mitigations (5)

Compare effects and implementation levels. Expand each entry for the additional classification and rationale.

  • Rate limiting and query quotas

    Technical
    Effect
    Preventive, Impact-limiting
    Implementation level
    Application, API & agents, Infrastructure
    Classification and rationale

    Complementary control typeOrganizational & process-based

    Primarily technical: Minimization, randomization, uniform processing, or query limits reduce information gained from repeated observations; complemented by binding workflows.

  • Watermarking

    Technical
    Effect
    Detective
    Implementation level
    Model & training, Application, API & agents
    Classification and rationale

    Primarily technical: Cryptographic or machine-verifiable properties protect confidentiality, integrity, or provenance.

  • Model output perturbation

    Technical
    Effect
    Preventive
    Implementation level
    Model & training, Application, API & agents
    Classification and rationale

    Primarily technical: Minimization, randomization, uniform processing, or query limits reduce information gained from repeated observations.

  • Access control and authentication

    Technical
    Effect
    Preventive
    Implementation level
    Application, API & agents, Infrastructure
    Classification and rationale

    Complementary control typeOrganizational & process-based

    Primarily technical: Machine-enforced identity, permission, or scope rules constrain unauthorized access and actions; complemented by binding workflows.

  • Monitor for extraction patterns

    Technical
    Effect
    Detective
    Implementation level
    Application, API & agents, Use & operations
    Classification and rationale

    Complementary control typeOrganizational & process-based

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators; complemented by binding workflows.

Framework mappings

Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.

OWASP LLM Top 10 LLM06:2026NIST AI RMF Section 2.10 · NISTAML.031MITRE ATLAS AML.T0040 · AML.T0044EU AI Act Article 55(1)(d)BSI R24BIML BIML78 model:5

Verified references (8)

Every reference states the framework, the exact location and the publishing organisation.

EU AI Act obligations related to this threat

The obligations from the obligations compass whose articles the catalogue maps to this threat. Each leads to its entry with core, action step and evidence; whether it applies to you depends on role and risk class.

  • Evaluate and mitigate systemic risks Art. 55 · applies from 2 August 2025

    Statutory roles: Provider of a general-purpose AI model. Classes: Model with systemic risk.

    This reference is a review prompt. A threat match alone does not establish applicability.

All 62 obligations by role, risk class and deadline in the obligations compass

More entries from the topic group Attacks on the Running Model and Service.

Adversarial Inputs Attackers alter an input in a way that is minimal and imperceptible to humans, so that the AI model classifies it incorrectly. Model Denial of Service Attackers overload the AI model with especially expensive requests until the service slows down or fails. Cost Harvesting / Repurposing Attackers use a company's paid AI services without authorisation and at its expense, for example with stolen access keys.

Matching use cases

This list shows where to review this threat, rather than how high the risk is.

Typical for:

For 43 further use cases, relevance depends on the system design, for example whether it accepts queries directly through an interface or is accessible over a network. Show these use cases

Frequently asked questions

What is Model Theft?
Attackers create a working copy of a proprietary AI model, either by querying it at scale or by breaking into the infrastructure. They steal the intellectual property without ever touching the original.
Which mitigations help against Model Theft?
The catalogue lists 5 mitigations, each with control type, effect and implementation level. First among them are Rate limiting and query quotas, Watermarking and Model output perturbation.
Which EU AI Act obligations does Model Theft touch?
The catalogue maps this threat to Art. 55 of the EU AI Act. In the obligations compass this leads to: Evaluate and mitigate systemic risks. Whether an obligation applies depends on role, risk class and deadline.

Assess this threat in your own system

The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.

Start the live demo See the free modules

Cite this entry

For reports, policies or internal documents; the link leads directly to this entry.

“Model Theft”. Versatile AI Risk Assessment, AI threat catalogue, as of October 2026.
https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/model-theft/

← Back to the full catalogue