AI threat cataloguePrivacy and Data Leakage Production

Sensitive Information Disclosure

An AI system discloses confidential information without anyone intending it: trade secrets, personal data, credentials or internal documents surface in the model's answers.

As of: July 2026 · Catalogue version 2026.07.17.3 · 4 mitigations · 18 verified sources

Description

Language models memorise parts of their training data and can later reproduce them verbatim or slightly altered. Confidential content reaches the answers along several paths: through memorised training data, through connected knowledge sources such as document stores and databases, or through inputs from other users. The disclosure is triggered by normal use, by deliberately crafted questions, or by prompt injection, meaning smuggled-in instructions that bypass protective filters. A second path arises in everyday work: employees enter confidential material into external AI services whose operators store the inputs and may use them for further training.

Possible impact

Trade secrets and customer data end up in the wrong hands; where personal data is involved, GDPR notification duties, fines and claims by data subjects can follow. Disclosed credentials open the door to follow-up attacks. Trust and reputation also suffer when customers learn that their data appears in answers given to third parties.

Example

Employees paste internal source code and meeting notes into a public AI chatbot to work faster. The content now sits with the provider and can flow into future model versions. One such case at a major electronics group became publicly known.

Recommended mitigations (4)

Every mitigation states its control type, effect, implementation level and the reason for the classification.

Data sanitization before trainingTechnical
Effect
Preventive
Implementation level
Data
Reason for the classification
“Data sanitization before training” is primarily technical: System-enforced inspection, transformation, or blocking rules stop or neutralize disallowed content before further processing.
Output filtering for sensitive patternsTechnical
Effect
Preventive
Implementation level
Data, Application, API & agents
Reason for the classification
“Output filtering for sensitive patterns” is primarily technical: System-enforced inspection, transformation, or blocking rules stop or neutralize disallowed content before further processing.
PII detection and redactionTechnical
Effect
Preventive, Detective
Implementation level
Data, Application, API & agents
Reason for the classification
“PII detection and redaction” is primarily technical: System-enforced inspection, transformation, or blocking rules stop or neutralize disallowed content before further processing.
Strict access controlsTechnical
Effect
Preventive
Implementation level
Data, Application, API & agents
Complementary control type
Organizational & process-based
Reason for the classification
“Strict access controls” is primarily technical: Machine-enforced identity, permission, or scope rules constrain unauthorized access and actions; complemented by binding workflows.

Framework mappings

Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.

OWASP LLM Top 10 LLM02:2025NIST AI RMF Section 2.10 · Section 2.4 · Section 2.9 · MEASURE 2.10 · NISTAML.032 · NISTAML.035 · NISTAML.038MITRE ATLAS AML.T0057EU AI Act Article 13(1), 13(3)(b)(ii), (iv), (v) · Article 55(1)(b)GDPR Article 25(1)–(2) · EDPB Opinion 28/2024, Section 3.2BSI R2 · R25BIML BIML-LLM inference:10 · BIML-LLM raw:5 · BIML78 inference:5

Verified references (18)

Every reference states the framework, the exact location and the publishing organisation.

OWASP LLM Top 10 LLM02:2025 Sensitive Information DisclosureLLM02:2025 Sensitive Information Disclosure, official category page OWASP Foundation Original
NIST AI RMF Section 2.10 Intellectual PropertySection 2.10, p. 11 National Institute of Standards and Technology (NIST)Original
NIST AI RMF Section 2.4 Data PrivacySection 2.4, pp. 7–8 National Institute of Standards and Technology (NIST)Original
NIST AI RMF Section 2.9 Information SecuritySection 2.9, pp. 10–11 National Institute of Standards and Technology (NIST)Original
NIST AI RMF MEASURE 2.10 MEASURE 2.10MEASURE 2.10, p. 30 National Institute of Standards and Technology (NIST)Original
NIST AI RMF NISTAML.032 ReconstructionTaxonomy Index, p. x; Section 2.4.1, pp. 28–29 National Institute of Standards and Technology (NIST)Original
NIST AI RMF NISTAML.035 Prompt ExtractionTaxonomy Index, p. xi; Section 3.3.2, pp. 46–47; Glossary, p. 111 National Institute of Standards and Technology (NIST)Original
NIST AI RMF NISTAML.038 Data ExtractionTaxonomy Index, p. xi; Section 3.3.2, pp. 46–47 National Institute of Standards and Technology (NIST)Original
MITRE ATLAS AML.T0057 LLM Data LeakageATLAS.yaml technique object with id AML.T0057 (pinned release v5.6.0) MITRE Original
EU AI Act Article 13(1), 13(3)(b)(ii), (iv), (v) Transparency and provision of information to deployersArticle 13(1), 13(3)(b)(ii), (iv), (v) European Union (EUR-Lex)Original
EU AI Act Article 55(1)(b) Obligations of providers of general-purpose AI models with systemic riskArticle 55(1)(b) European Union (EUR-Lex)Original
GDPR Article 25(1)–(2) Data protection by design and by defaultArticle 25(1) and 25(2) European Union (EUR-Lex)Original
GDPR EDPB Opinion 28/2024, Section 3.2 EDPB Opinion 28/2024 on certain data protection aspects related to the processing of personal data in the context of AI modelsSection 3.2, especially paragraph 43 and paragraphs 44–58, pp. 14–18 (anonymity; extraction/model attacks) European Data Protection Board (EDPB)Original
BSI R2 Fehlende Vertraulichkeit eingegebener Daten (Text, Bild, Video)Kap. 4, R2, p. 14 Bundesamt für Sicherheit in der Informationstechnik (BSI)Original
BSI R25 Extraktion von Kommunikationsdaten und hinterlegten Informationen (Text, Bild, Video)Kap. 4, R25, p. 30 Bundesamt für Sicherheit in der Informationstechnik (BSI)Original
BIML BIML-LLM inference:10 User RiskPDF p. 19, [inference:10:user risk] Berryville Institute of Machine Learning (BIML)Original
BIML BIML-LLM raw:5 Data ConfidentialityPDF p. 15, [raw:5:data confidentiality] Berryville Institute of Machine Learning (BIML)Original
BIML BIML78 inference:5 User RiskPDF p. 20, [inference:5:user risk] Berryville Institute of Machine Learning (BIML)Original

Terms on this page

Glossary terms that occur in this entry. Every link leads to the full explanation.

Prompt Injection Manipulated input or planted content redirects a language model.

More entries from the topic group Privacy and Data Leakage.

Assess this threat in your own system

The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.

Start the live demo See the free modules

Cite this entry

For reports, policies or internal documents; the link leads directly to this entry.

“Sensitive Information Disclosure”. Versatile AI Risk Assessment, AI threat catalogue, as of July 2026.
https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/sensitive-information-disclosure/

← Back to the full catalogue