Versatile AI Risk Assessment

AI threat catalogue Application and Integration Security Production

Middleware Exploits (AI Framework Attacks)

AI applications are built on frameworks, libraries and middleware, the connecting software between model and application. Vulnerabilities or manipulated packages in these building blocks open the door to attackers without the model itself being attacked.

As of: October 2026 · Catalogue version 2026.10.24.2 · 7 mitigations · 10 verified sources

Description

Between the model and the finished application sits a software layer of AI frameworks (such as LangChain or LlamaIndex), vector database drivers and inference servers. These building blocks pull in long chains of their own dependencies, often hundreds of packages from open sources. Attackers exploit known vulnerabilities in such components or smuggle manipulated packages into the supply chain, for example through name confusion during installation (dependency confusion) or through initially harmless packages that later receive a malicious update. The effect propagates along the dependency chain: a single vulnerable library is enough, because it runs with far-reaching privileges at the core of the application.

Possible impact

Attackers establish themselves inside the application: they read credentials, keys and the data flowing between model and application, manipulate answers or execute their own code. Because the same frameworks are used across industries, a single vulnerability hits many organisations at once; without an inventory of its dependencies (SBOM, a software bill of materials), an organisation cannot tell whether it is affected.

Example

A development team installs the current version of a popular AI framework. Through the package manager, a counterfeit companion library with the same name silently ends up on the systems and sends credentials and SSH keys to a remote server.

Recommended mitigations (7)

Compare effects and implementation levels. Expand each entry for the additional classification and rationale.

  • Dependency vulnerability scanning

    Technical
    Effect
    Detective
    Implementation level
    Application, API & agents, Supply chain, Use & operations
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

  • Pinned dependency versions

    Technical
    Effect
    Preventive
    Implementation level
    Application, API & agents, Supply chain
    Classification and rationale

    Primarily technical: Safe formats, restrictive defaults, or protective layers reduce unsafe execution paths and exploitable attack surface.

  • SBOM (software bill of materials) maintenance

    Organizational & process-based
    Effect
    Detective
    Implementation level
    Organization, Supply chain, Use & operations
    Classification and rationale

    Complementary control typeTechnical

    Primarily organizational and process-based: Defined selection, operating, or lifecycle procedures make the control binding and repeatable; complemented by technical implementation.

  • Framework update policies

    Governance & compliance
    Effect
    Preventive
    Implementation level
    Organization, Supply chain, Use & operations
    Classification and rationale

    Complementary control typeOrganizational & process-based

    Primarily a governance and compliance control: Binding rules, control objectives, or oversight define permitted use and accountability; complemented by binding workflows.

  • Security monitoring of middleware components

    Technical
    Effect
    Detective
    Implementation level
    Application, API & agents, Supply chain, Use & operations
    Classification and rationale

    Complementary control typeOrganizational & process-based

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators; complemented by binding workflows.

  • Isolation between middleware and application

    Technical
    Effect
    Preventive, Impact-limiting
    Implementation level
    Application, API & agents, Infrastructure
    Classification and rationale

    Primarily technical: Architectural and runtime boundaries separate data, execution, or tenants and limit the spread of failures.

  • Regular pentest of integrated stack

    Organizational & process-based
    Effect
    Detective
    Implementation level
    Application, API & agents, Infrastructure, Use & operations
    Classification and rationale

    Complementary control typePeople & competence, Technical

    Primarily organizational and process-based: A planned, repeatable assessment with ownership and documented follow-up creates the protective effect; complemented by human expertise and judgment as well as technical implementation.

Framework mappings

Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.

OWASP LLM Top 10 LLM04:2026NIST AI RMF Section 2.12 · MAP 4.1 · NISTAML.05MITRE ATLAS AML.T0010 · AML.T0049 · AML.T0109EU AI Act Article 25(4) · Article 55(1)(d) · Article 9(1), 9(2)(a), 9(2)(d)

Verified references (10)

Every reference states the framework, the exact location and the publishing organisation.

EU AI Act obligations related to this threat

The obligations from the obligations compass whose articles the catalogue maps to this threat. Each leads to its entry with core, action step and evidence; whether it applies to you depends on role and risk class.

  • Establish and operate a risk management system Art. 9 · applies from Annex III: 2 December 2027 · Annex I: 2 August 2028

    Statutory roles: Provider, Product manufacturer. Classes: High-risk under Annex III, High-risk as a product or safety component.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Support downstream providers Art. 25 Abs. 2, Art. 25 Abs. 4 · applies from Annex III: 2 December 2027 · Annex I: 2 August 2028 · amended by Reg. 2026/1744

    Statutory roles: Provider, Provider of a general-purpose AI model. Classes: High-risk under Annex III, High-risk as a product or safety component.

    Regulation (EU) 2026/1744 spelled out the initial provider duty to cooperate and at the same time created an exception: it does not apply where the initial provider has clearly specified that their system must not be turned into a high-risk system. Infringements of Art. 25(2) and (4) are now expressly subject to fines.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Evaluate and mitigate systemic risks Art. 55 · applies from 2 August 2025

    Statutory roles: Provider of a general-purpose AI model. Classes: Model with systemic risk.

    This reference is a review prompt. A threat match alone does not establish applicability.

All 62 obligations by role, risk class and deadline in the obligations compass

More entries from the topic group Application and Integration Security.

Insecure Output Handling The application passes an AI model's answers to downstream systems without checks, for example to databases, web pages or command lines. Insecure Tool Design Many AI applications act on their own through connected tools, for example sending emails or querying databases. Application Vulnerabilities The software around the AI model, meaning its web interface, APIs and databases, has the same weaknesses as any other application.

Matching use cases

This list shows where to review this threat, rather than how high the risk is.

Belongs on the checklist for every AI system. What to review for every AI system

Frequently asked questions

What is Middleware Exploits (AI Framework Attacks)?
AI applications are built on frameworks, libraries and middleware, the connecting software between model and application. Vulnerabilities or manipulated packages in these building blocks open the door to attackers without the model itself being attacked.
Which mitigations help against Middleware Exploits (AI Framework Attacks)?
The catalogue lists 7 mitigations, each with control type, effect and implementation level. First among them are Dependency vulnerability scanning, Pinned dependency versions and SBOM (software bill of materials) maintenance.
Which EU AI Act obligations does Middleware Exploits (AI Framework Attacks) touch?
The catalogue maps this threat to Art. 9, Art. 25 Abs. 2 and Art. 55 of the EU AI Act. In the obligations compass this leads to: Establish and operate a risk management system, Support downstream providers and Evaluate and mitigate systemic risks. Whether an obligation applies depends on role, risk class and deadline.

Assess this threat in your own system

The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.

Start the live demo See the free modules

Cite this entry

For reports, policies or internal documents; the link leads directly to this entry.

“Middleware Exploits (AI Framework Attacks)”. Versatile AI Risk Assessment, AI threat catalogue, as of October 2026.
https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/middleware-exploits/

← Back to the full catalogue