Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
Summary
Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model (2019) introduced MuZero, which learns a model of the environment and then plans in that model, without being given the official rules or a perfect simulator. The authors show strong results on Atari, Go, chess, and shogi, extending the AlphaZero line to settings where the dynamics are not known in advance.
What it introduced: a model-based planning agent that trains a dynamics model, a value head, and a policy head together, then uses tree search in the learned model rather than in a hand-specified simulator.
What it led to: later model-based reinforcement learning and planning work that treats a learned world model as the place where search happens. Live note: 1712-01815-alphazero.
Use it when you want planning without an official game engine, after reading AlphaZero. Where to read it: https://arxiv.org/abs/1911.08265 (v2 (21 Feb 2020)).
The paper is posted on arXiv under the arXiv non-exclusive distribution license. This note links to it rather than reproducing it. The note itself is CC BY 4.0.
License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.
License: CC-BY-4.0 · Pointer by iinventedlight (AI-assisted). Source: Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, and 9 coauthors (listed on the arXiv abs page), Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model (arXiv non-exclusive distribution license). This note links to the paper rather than reproducing it.
Cite this
'iinventedlight (AI-assisted)'. 2026. "Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model." iinventedlight Library, iil:note:0hyxsakb, version of 2026-11-12. https://iinventedlight.com/library/notes/1911-08265-muzero/. License: CC-BY-4.0.
@misc{iil_note_0hyxsakb,
author = {{'iinventedlight (AI-assisted)'}},
title = {{Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model}},
howpublished = {iinventedlight Library},
year = {2026},
date = {2026-11-12},
url = {https://iinventedlight.com/library/notes/1911-08265-muzero/},
note = {Permanent ID iil:note:0hyxsakb. License: CC-BY-4.0.},
}
Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.
For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/1911-08265-muzero.md.