Versatile AI Risk Assessment

AI threat catalogue Attacks on the Running Model and Service Production

Model Reconnaissance

Attackers gather information about a deployed model, such as its design, the model family used and its capabilities, to prepare more targeted attacks later. This is the reconnaissance phase before theft or evasion.

As of: October 2026 · Catalogue version 2026.10.24.2 · 6 mitigations · 12 verified sources

Description

Before a targeted attack, attackers scout out the deployed model. Through the normal interface they probe which model family and version lies behind it (fingerprinting), which architecture is likely in use, and what capabilities and output categories the model has. To do so they analyse answers, accompanying metadata, verbose error messages or publicly accessible artefacts. This reconnaissance is usually not an end in itself but the precursor to model theft or to adversarial inputs that are then tailored precisely.

Possible impact

The immediate damage is small, but the insights gained make follow-on attacks considerably more effective and harder to defend against. Once the underlying model family is identified, known weaknesses of that family can be exploited in a targeted way, especially since many services build on the same base models. At the same time, internal model and configuration details can leak as trade secrets.

Example

An attacker sends an AI service a series of systematic test questions and analyses phrasing, response times and error messages. From this they infer the base model in use and aim their next attack squarely at its known weaknesses.

Recommended mitigations (6)

Compare effects and implementation levels. Expand each entry for the additional classification and rationale.

  • Query pattern anomaly detection

    Technical
    Effect
    Detective
    Implementation level
    Application, API & agents, Infrastructure
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

  • Output randomization to prevent fingerprinting

    Technical
    Effect
    Preventive
    Implementation level
    Application, API & agents, Infrastructure
    Classification and rationale

    Primarily technical: Minimization, randomization, uniform processing, or query limits reduce information gained from repeated observations.

  • Minimize model metadata in responses

    Technical
    Effect
    Preventive
    Implementation level
    Application, API & agents, Infrastructure
    Classification and rationale

    Primarily technical: Minimization, randomization, uniform processing, or query limits reduce information gained from repeated observations.

  • Rate limiting per session

    Technical
    Effect
    Preventive, Impact-limiting
    Implementation level
    Application, API & agents, Infrastructure
    Classification and rationale

    Complementary control typeOrganizational & process-based

    Primarily technical: Minimization, randomization, uniform processing, or query limits reduce information gained from repeated observations; complemented by binding workflows.

  • Honeypot queries for attack detection

    Technical
    Effect
    Detective
    Implementation level
    Application, API & agents, Infrastructure, Use & operations
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

  • Access controls and authentication

    Technical
    Effect
    Preventive
    Implementation level
    Application, API & agents, Infrastructure
    Classification and rationale

    Complementary control typeOrganizational & process-based

    Primarily technical: Machine-enforced identity, permission, or scope rules constrain unauthorized access and actions; complemented by binding workflows.

Framework mappings

Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.

OWASP LLM Top 10 LLM02:2026 · LLM06:2026NIST AI RMF NISTAML.031MITRE ATLAS AML.T0002 · AML.T0014 · AML.T0040EU AI Act Article 55(1)(d)BSI R12 · R25BIML BIML-LLM input:3 · BIML78 assembly:6 · BIML78 inference:3

Verified references (12)

Every reference states the framework, the exact location and the publishing organisation.

EU AI Act obligations related to this threat

The obligations from the obligations compass whose articles the catalogue maps to this threat. Each leads to its entry with core, action step and evidence; whether it applies to you depends on role and risk class.

  • Evaluate and mitigate systemic risks Art. 55 · applies from 2 August 2025

    Statutory roles: Provider of a general-purpose AI model. Classes: Model with systemic risk.

    This reference is a review prompt. A threat match alone does not establish applicability.

All 62 obligations by role, risk class and deadline in the obligations compass

More entries from the topic group Attacks on the Running Model and Service.

Model Theft Attackers create a working copy of a proprietary AI model, either by querying it at scale or by breaking into the infrastructure. Adversarial Inputs Attackers alter an input in a way that is minimal and imperceptible to humans, so that the AI model classifies it incorrectly. Model Denial of Service Attackers overload the AI model with especially expensive requests until the service slows down or fails.

Matching use cases

This list shows where to review this threat, rather than how high the risk is.

Typical for:

For 44 further use cases, relevance depends on the system design, for example whether it accepts queries directly through an interface or is also available to people outside your organisation. Show these use cases

Frequently asked questions

What is Model Reconnaissance?
Attackers gather information about a deployed model, such as its design, the model family used and its capabilities, to prepare more targeted attacks later. This is the reconnaissance phase before theft or evasion.
Which mitigations help against Model Reconnaissance?
The catalogue lists 6 mitigations, each with control type, effect and implementation level. First among them are Query pattern anomaly detection, Output randomization to prevent fingerprinting and Minimize model metadata in responses.
Which EU AI Act obligations does Model Reconnaissance touch?
The catalogue maps this threat to Art. 55 of the EU AI Act. In the obligations compass this leads to: Evaluate and mitigate systemic risks. Whether an obligation applies depends on role, risk class and deadline.

Assess this threat in your own system

The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.

Start the live demo See the free modules

Cite this entry

For reports, policies or internal documents; the link leads directly to this entry.

“Model Reconnaissance”. Versatile AI Risk Assessment, AI threat catalogue, as of October 2026.
https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/model-reconnaissance/

← Back to the full catalogue