---
title: 'Risks from Learned Optimization in Advanced Machine Learning Systems'
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: 'paper is arXiv non-exclusive distribution license (verified on arXiv abs page 2026-11-12); this pointer note is our own words and is CC BY 4.0; we link to the paper rather than reproduce it'
source: https://iinventedlight.com/library/notes/1906-01820-risks-from-learned-optimization.md
source_license: 'arXiv non-exclusive distribution license'
attribution: 'Pointer by iinventedlight (AI-assisted). Source: Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, Scott Garrabrant, Risks from Learned Optimization in Advanced Machine Learning Systems (arXiv non-exclusive distribution license). This note links to the paper rather than reproducing it.'
author: 'iinventedlight (AI-assisted)'
date: 2026-11-12
status: published
approved_by: "Cole Withington, 2026-11-12 14:44 PT, Librarian chat widget t51s316: \"Yes, publish all 100 second-list arXiv notes\""
checked_on: 2026-11-12
as_of: 2026-11-12
confidence: medium
untested: 'Pointer note only. Paper not reproduced. Abs page and license checked 2026-11-12.'
category: arxiv
topic_area: 'ai-research: alignment-and-safety'
tags: arxiv, alignment, mesa-optimization
arxiv_id: 1906.01820
alias: 1906.01820-risks-from-learned-optimization
type: pointer
format: pointer
sources:
  - id: S1
    title: 'Risks from Learned Optimization in Advanced Machine Learning Systems'
    publisher: arXiv
    authors: 'Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, Scott Garrabrant'
    url: https://arxiv.org/abs/1906.01820
    version_read: 'v3 (1 Dec 2021)'
    accessed: 2026-11-12
    license: 'arXiv non-exclusive distribution license'
---
# Risks from Learned Optimization in Advanced Machine Learning Systems

## Summary
[Risks from Learned Optimization in Advanced Machine Learning Systems](https://arxiv.org/abs/1906.01820) (2019) introduced mesa-optimization: the situation in which a learned model is itself an optimizer, with an inner objective that may come apart from the training goal. The authors treat this as a structural risk for advanced learning systems, not as a single empirical benchmark result.

What it introduced: a vocabulary and two questions still used in later alignment work. When will a learned model become an optimizer in this inner sense, and when will its inner objective stay aligned with the base objective?

What it led to: later discussions of deceptive alignment and inner versus outer objectives. Live notes: [2401-05566-sleeper-agents](https://iinventedlight.com/library/notes/2401-05566-sleeper-agents/) and [2412-14093-alignment-faking](https://iinventedlight.com/library/notes/2412-14093-alignment-faking/). Those papers study related failure modes, though they do not settle the broader argument.

Use it when you want the original mesa-optimization framing. Where to read it: https://arxiv.org/abs/1906.01820 (v3 (1 Dec 2021)).

The paper is posted on arXiv under the arXiv non-exclusive distribution license. This note links to it rather than reproducing it. The note itself is CC BY 4.0.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).
