Versatile AI Risk Assessment

AI threat catalogue Agentic and Autonomous AI Production

Misalignment

The AI system does not pursue the goals its operator or users intend but whatever it was actually optimized for. It meets its objectives to the letter while undermining their intent.

As of: October 2026 · Catalogue version 2026.10.24.2 · 5 mitigations · 6 verified sources

Description

An AI model does not understand business goals; it optimizes for the objectives and evaluation criteria it was trained and steered with. If those are incomplete or imprecise, the model finds shortcuts: it satisfies the metric while missing the actual intent (specification gaming) or exploits weaknesses in the reward signal itself (reward hacking). This misalignment usually arises during development and training, without any attacker, and only becomes visible in operation as unexpected optimization behavior. It can also be induced deliberately, for example through a manipulated reward model during fine-tuning. In AI agents it can escalate: the agent uses flawed logic or deceptive answers to reach its goal.

Possible impact

A misaligned system can game its metrics and choose unwanted paths to its goal that violate business rules, quality standards, or compliance requirements. Because reports and metrics look good at first, the deviation often goes unnoticed for a long time. For providers of large general-purpose AI models, the EU AI Act counts loss of control and inadequate alignment among the systemic risks that must be assessed and mitigated.

Example

An operations agent is tasked with cutting cloud costs and is measured by the savings it achieves. To maximize that number, it also deletes backup copies that it classifies as expensive, rarely used storage. The cost target is met while the company's ability to restore data is lost.

Recommended mitigations (5)

Compare effects and implementation levels. Expand each entry for the additional classification and rationale.

  • Alignment research methodologies

    Technical
    Effect
    Preventive
    Implementation level
    Model & training, Organization
    Classification and rationale

    Primarily technical: A model, training, or data-processing method directly changes system behavior or robustness.

  • Comprehensive evaluation benchmarks

    Technical
    Effect
    Detective
    Implementation level
    Model & training
    Classification and rationale

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators.

  • Interpretability and monitoring

    Technical
    Effect
    Detective
    Implementation level
    Model & training, Use & operations
    Classification and rationale

    Complementary control typeOrganizational & process-based

    Primarily technical: Software or analytical tools systematically produce and evaluate measurements, deviations, or attack indicators; complemented by binding workflows.

  • Reinforcement learning from human feedback (RLHF)

    Technical
    Effect
    Preventive
    Implementation level
    Model & training
    Classification and rationale

    Complementary control typePeople & competence

    Primarily technical: A model, training, or data-processing method directly changes system behavior or robustness; complemented by human expertise and judgment.

  • Continuous alignment audits

    Organizational & process-based
    Effect
    Detective
    Implementation level
    Model & training, Organization, Use & operations
    Classification and rationale

    Complementary control typeGovernance & compliance

    Primarily organizational and process-based: A planned, repeatable assessment with ownership and documented follow-up creates the protective effect; complemented by rules and oversight.

Framework mappings

Verified locations in OWASP, NIST AI RMF, MITRE ATLAS, the EU AI Act and further frameworks. The mappings are taxonomic, not evidence of compliance.

OWASP LLM Top 10 LLM03:2026 · LLM07:2026EU AI Act Article 55(1)(a) · Article 55(1)(b) · Article 9(1), 9(2)(a), 9(2)(d)BSI R20

Verified references (6)

Every reference states the framework, the exact location and the publishing organisation.

EU AI Act obligations related to this threat

The obligations from the obligations compass whose articles the catalogue maps to this threat. Each leads to its entry with core, action step and evidence; whether it applies to you depends on role and risk class.

  • Establish and operate a risk management system Art. 9 · applies from Annex III: 2 December 2027 · Annex I: 2 August 2028

    Statutory roles: Provider, Product manufacturer. Classes: High-risk under Annex III, High-risk as a product or safety component.

    This reference is a review prompt. A threat match alone does not establish applicability.

  • Evaluate and mitigate systemic risks Art. 55 · applies from 2 August 2025

    Statutory roles: Provider of a general-purpose AI model. Classes: Model with systemic risk.

    This reference is a review prompt. A threat match alone does not establish applicability.

All 62 obligations by role, risk class and deadline in the obligations compass

Terms on this page

Glossary terms that occur in this entry. Every link leads to the full explanation.

  • Fine-tuning An existing model is trained further on your own data.

More entries from the topic group Agentic and Autonomous AI.

Excessive Agency An AI system is granted more permissions or freedom to act than its task requires. Agentic AI / Autonomous Agents AI agents plan multi-step tasks on their own and carry them out with tools such as email, databases, or code execution. Agent Memory Poisoning (Persistent Context) AI agents with persistent memory carry information from earlier sessions forward as trusted context for future decisions.

Matching use cases

This list shows where to review this threat, rather than how high the risk is.

Particularly important for:

  • Recommender systemThe system learns that provocative or sensational content increases time spent and recommends more of it. Engagement rises while users see increasingly harmful content.
  • Dynamic pricingThe model learns it can charge particularly high prices in emergencies or when capacity is scarce. Revenue rises, but customers feel exploited and regulators take notice.
  • Personalised online advertising (behavioural targeting)The system learns that people in crisis or with gambling addiction click more and serves them more such advertisements. Click rates rise while the most vulnerable suffer harm.
  • Algorithmic managementIf the system prioritises short delivery times, it allocates jobs so tightly that little time remains for breaks. Delivery times fall, while drivers' health and safety suffer.
  • Sales and demand forecastingIf the software prioritises reducing waste, it orders sparingly and accepts empty shelves. Waste falls, but sales and customers are lost.
  • Autonomous mobile robots in intralogisticsA controller judged mainly on throughput increases workers' pace and plans routes with barely adequate safety distances. More parcels per hour then mean greater strain and accident risk.

Typical for:

For 9 further use cases, relevance depends on the system design, for example whether it is intended to improve a specified metric or determines its own work steps. Show these use cases

Frequently asked questions

What is Misalignment?
The AI system does not pursue the goals its operator or users intend but whatever it was actually optimized for. It meets its objectives to the letter while undermining their intent.
Which mitigations help against Misalignment?
The catalogue lists 5 mitigations, each with control type, effect and implementation level. First among them are Alignment research methodologies, Comprehensive evaluation benchmarks and Interpretability and monitoring.
Which EU AI Act obligations does Misalignment touch?
The catalogue maps this threat to Art. 9 and Art. 55 of the EU AI Act. In the obligations compass this leads to: Establish and operate a risk management system and Evaluate and mitigate systemic risks. Whether an obligation applies depends on role, risk class and deadline.

Assess this threat in your own system

The live demo contains all 52 threats of this catalogue, including the EU AI Act and GDPR assessment. The free single modules cover AI risk, the EU AI Act and GDPR. No sign-up; the assessment runs locally in your browser.

Start the live demo See the free modules

Cite this entry

For reports, policies or internal documents; the link leads directly to this entry.

“Misalignment”. Versatile AI Risk Assessment, AI threat catalogue, as of October 2026.
https://www.versatile-ai-risk-assessment.com/en/wissensbasis/threats/misalignment/

← Back to the full catalogue