AI threat catalogueApplication and Integration SecurityProduction
Middleware Exploits (AI Framework Attacks)
AI applications are built on frameworks, libraries and middleware, the connecting software between model and application. Vulnerabilities or manipulated packages in these building blocks open the door to attackers without the model itself being attacked.
Description
Between the model and the finished application sits a software layer of AI frameworks (such as LangChain or LlamaIndex), vector database drivers and inference servers. These building blocks pull in long chains of their own dependencies, often hundreds of packages from open sources. Attackers exploit known vulnerabilities in such components or smuggle manipulated packages into the supply chain, for example through name confusion during installation (dependency confusion) or through initially harmless packages that later receive a malicious update. The effect propagates along the dependency chain: a single vulnerable library is enough, because it runs with far-reaching privileges at the core of the application.
Possible impact
Attackers establish themselves inside the application: they read credentials, keys and the data flowing between model and application, manipulate answers or execute their own code. Because the same frameworks are used across industries, a single vulnerability hits many organisations at once; without an inventory of its dependencies (SBOM, a software bill of materials), an organisation cannot tell whether it is affected.
Example
A development team installs the current version of a popular AI framework. Through the package manager, a counterfeit companion library with the same name silently ends up on the systems and sends credentials and SSH keys to a remote server.
Recommended mitigations (7)
Every mitigation states its control type, effect, implementation level and the reason for the classification.
Dependency vulnerability scanningTechnical
- Effect
- Detective
- Implementation level
- Application, API & agents, Supply chain, Use & operations
- Reason for the classification
- “Dependency vulnerability scanning” is primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.
Pinned dependency versionsTechnical
- Effect
- Preventive
- Implementation level
- Application, API & agents, Supply chain
- Reason for the classification
- “Pinned dependency versions” is primarily technical: Safe formats, restrictive defaults, or protective layers reduce unsafe execution paths and exploitable attack surface.
SBOM (software bill of materials) maintenanceOrganizational & process-based
- Effect
- Detective
- Implementation level
- Organization, Supply chain, Use & operations
- Complementary control type
- Technical
- Reason for the classification
- “SBOM (software bill of materials) maintenance” is primarily organizational and process-based: Defined selection, operating, or lifecycle procedures make the control binding and repeatable; complemented by technical implementation.
Framework update policiesGovernance & compliance
- Effect
- Preventive
- Implementation level
- Organization, Supply chain, Use & operations
- Complementary control type
- Organizational & process-based
- Reason for the classification
- “Framework update policies” is primarily a governance and compliance control: Binding rules, control objectives, or oversight define permitted use and accountability; complemented by binding workflows.
Security monitoring of middleware componentsTechnical
- Effect
- Detective
- Implementation level
- Application, API & agents, Supply chain, Use & operations
- Complementary control type
- Organizational & process-based
- Reason for the classification
- “Security monitoring of middleware components” is primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators; complemented by binding workflows.
Isolation between middleware and applicationTechnical
- Effect
- Preventive, Impact-limiting
- Implementation level
- Application, API & agents, Infrastructure
- Reason for the classification
- “Isolation between middleware and application” is primarily technical: Architectural and runtime boundaries separate data, execution, or tenants and limit the spread of failures.
Regular pentest of integrated stackOrganizational & process-based
- Effect
- Detective
- Implementation level
- Application, API & agents, Infrastructure, Use & operations
- Complementary control type
- People & competence, Technical
- Reason for the classification
- “Regular pentest of integrated stack” is primarily organizational and process-based: A planned, repeatable assessment with ownership and documented follow-up creates the protective effect; complemented by human expertise and judgment as well as technical implementation.
Framework mappings
Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.
Verified references (9)
Every reference states the framework, the exact location and the publishing organisation.
- OWASP LLM Top 10 LLM03:2025 Supply ChainLLM03:2025 Supply Chain, official category page OWASP FoundationOriginal
- NIST AI RMF Section 2.12 Value Chain and Component IntegrationSection 2.12, p. 12 National Institute of Standards and Technology (NIST)Original
- NIST AI RMF MAP 4.1 MAP 4.1MAP 4.1, p. 27 National Institute of Standards and Technology (NIST)Original
- NIST AI RMF NISTAML.05 Supply Chain AttacksTaxonomy Index, pp. x–xi; Section 3.2, pp. 41–43 National Institute of Standards and Technology (NIST)Original
- MITRE ATLAS AML.T0010 AI Supply Chain CompromiseATLAS.yaml technique object with id AML.T0010 (pinned release v5.6.0) MITREOriginal
- MITRE ATLAS AML.T0049 Exploit Public-Facing ApplicationATLAS.yaml technique object with id AML.T0049 (pinned release v5.6.0) MITREOriginal
- EU AI Act Article 25(4) Responsibilities along the AI value chainArticle 25(4) European Union (EUR-Lex)Original
- EU AI Act Article 55(1)(d) Obligations of providers of general-purpose AI models with systemic riskArticle 55(1)(d) European Union (EUR-Lex)Original
- EU AI Act Article 9(1), 9(2)(a), 9(2)(d) Risk management systemArticle 9(1), 9(2)(a), 9(2)(d), read with Article 9(3) European Union (EUR-Lex)Original
Related threats
More entries from the topic group Application and Integration Security.
Assess this threat in your own system
The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.
Cite this entry
For reports, policies or internal documents; the link leads directly to this entry.
“Middleware Exploits (AI Framework Attacks)”. Versatile AI Risk Assessment, AI threat catalogue, as of July 2026. https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/middleware-exploits/