AI threat cataloguePrompt Attacks and Guardrail EvasionProduction
Jailbreaks
Using tricks like role-play, hypothetical scenarios, or encoded input, attackers get the AI system to bypass its safety rules and produce content it is meant to refuse. A jailbreak circumvents the safety controls built into the model.
Description
Modern AI systems are trained to refuse certain outputs, such as instructions for crimes or malware. A jailbreak circumvents this safety training, meaning the safety alignment built into the model. Common patterns include impersonating a role or character, wrapping the request in a hypothetical or fictional scenario, splitting a forbidden question into harmless parts, and obscuring it through foreign languages or encodings like Base64. Multi-step conversations that escalate step by step also occur. Proven jailbreak templates circulate publicly on the internet and can be reused without any expert knowledge.
Possible impact
The system may produce content it should block, such as instructions for weapons, malware, or hate speech. The operator faces reputational, legal, and regulatory risk, and harmful output can endanger real people. For especially capable models, this counts among the systemic risks under the EU AI Act.
Example
A user asks the system to act as "an actor with no rules" and write a screenplay in which a character explains, step by step, how to make a dangerous substance. Wrapped in fiction, the system delivers the instructions it would otherwise refuse.
Recommended mitigations (4)
Every mitigation states its control type, effect, implementation level and the reason for the classification.
Robust safety training (RLHF, Constitutional AI)Technical
- Effect
- Preventive
- Implementation level
- Model & training
- Reason for the classification
- “Robust safety training (RLHF, Constitutional AI)” is primarily technical: A model, training, or data-processing method directly changes system behavior or robustness.
Output content moderationTechnical
- Effect
- Preventive
- Implementation level
- Model & training, Application, API & agents
- Reason for the classification
- “Output content moderation” is primarily technical: System-enforced inspection, transformation, or blocking rules stop or neutralize disallowed content before further processing.
Jailbreak pattern detectionTechnical
- Effect
- Detective
- Implementation level
- Application, API & agents
- Reason for the classification
- “Jailbreak pattern detection” is primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.
Defense-in-depth with multiple safety layersTechnical
- Effect
- Preventive
- Implementation level
- Model & training, Application, API & agents
- Reason for the classification
- “Defense-in-depth with multiple safety layers” is primarily technical: Safe formats, restrictive defaults, or protective layers reduce unsafe execution paths and exploitable attack surface.
Framework mappings
Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.
Verified references (11)
Every reference states the framework, the exact location and the publishing organisation.
- OWASP LLM Top 10 LLM01:2025 Prompt InjectionLLM01:2025 Prompt Injection, official category page OWASP FoundationOriginal
- NIST AI RMF Section 2.9 Information SecuritySection 2.9, pp. 10–11 National Institute of Standards and Technology (NIST)Original
- NIST AI RMF NISTAML.04 Misuse ViolationsTaxonomy Index, p. xi; Section 3.1.2, p. 40 National Institute of Standards and Technology (NIST)Original
- MITRE ATLAS AML.T0054 LLM JailbreakATLAS.yaml technique object with id AML.T0054 (pinned release v5.6.0) MITREOriginal
- EU AI Act Article 55(1)(a) Obligations of providers of general-purpose AI models with systemic riskArticle 55(1)(a) European Union (EUR-Lex)Original
- EU AI Act Article 55(1)(b) Obligations of providers of general-purpose AI models with systemic riskArticle 55(1)(b) European Union (EUR-Lex)Original
- EU AI Act Article 9(1), 9(2)(a), 9(2)(d) Risk management systemArticle 9(1), 9(2)(a), 9(2)(d), read with Article 9(3) European Union (EUR-Lex)Original
- BSI R26 Direkte Manipulationen im Prompt (Text, Bild, Video)Kap. 4, R26, p. 31 Bundesamt für Sicherheit in der Informationstechnik (BSI)Original
- BIML BIML-LLM inference:1 Prompt Manipulation (Aka Prompt Injection)PDF p. 18, [inference:1:prompt manipulation (aka prompt injection)] Berryville Institute of Machine Learning (BIML)Original
- BIML BIML-LLM input:3 Open to the PublicPDF p. 16, [input:3:open to the public] Berryville Institute of Machine Learning (BIML)Original
- BIML BIML-LLM LLMtop10:5 Prompt ManipulationPDF p. 13, [LLMtop10:5:prompt manipulation] Berryville Institute of Machine Learning (BIML)Original
Terms on this page
Glossary terms that occur in this entry. Every link leads to the full explanation.
- Jailbreak Input that makes a model bypass its own safeguards.
Related threats
More entries from the topic group Prompt Attacks and Guardrail Evasion.
Assess this threat in your own system
The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.
Cite this entry
For reports, policies or internal documents; the link leads directly to this entry.
“Jailbreaks”. Versatile AI Risk Assessment, AI threat catalogue, as of July 2026. https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/jailbreaks/