---
title: 'Scaling Language Models: Methods, Analysis & Insights from Training Gopher'
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: 'paper is arXiv non-exclusive distribution license (verified on arXiv abs page 2026-11-12); this pointer note is our own words and is CC BY 4.0; we link to the paper rather than reproduce it'
source: https://iinventedlight.com/library/notes/2112-11446-gopher.md
source_license: 'arXiv non-exclusive distribution license'
attribution: 'Pointer by iinventedlight (AI-assisted). Source: Jack W. Rae and 79 coauthors (listed on the arXiv abs page), Scaling Language Models: Methods, Analysis & Insights from Training Gopher (arXiv non-exclusive distribution license). This note links to the paper rather than reproducing it.'
author: 'iinventedlight (AI-assisted)'
date: 2026-11-12
status: published
approved_by: "Cole Withington, 2026-11-12 14:44 PT, Librarian chat widget t51s316: \"Yes, publish all 100 second-list arXiv notes\""
checked_on: 2026-11-12
as_of: 2026-11-12
confidence: medium
untested: 'Pointer note only. Paper not reproduced. Abs page and license checked 2026-11-12.'
category: arxiv
topic_area: 'ai-research: foundations'
tags: arxiv, foundations, language-models, scaling
arxiv_id: 2112.11446
alias: 2112.11446-gopher
type: pointer
format: pointer
sources:
  - id: S1
    title: 'Scaling Language Models: Methods, Analysis & Insights from Training Gopher'
    publisher: arXiv
    authors: 'Jack W. Rae and 79 coauthors (listed on the arXiv abs page)'
    url: https://arxiv.org/abs/2112.11446
    version_read: 'v2 (21 Jan 2022)'
    accessed: 2026-11-12
    license: 'arXiv non-exclusive distribution license'
---
# Scaling Language Models: Methods, Analysis & Insights from Training Gopher

## Summary
[Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://arxiv.org/abs/2112.11446) (2021) is DeepMind's study of Transformer language models trained across a wide range of scales, up to a 280 billion parameter model named Gopher. It became a main reference for later scale-and-data work at the same lab.

What it introduced: a carefully documented 280B dense language model, plus analysis of how quality and task results change as scale grows from tens of millions of parameters up to Gopher.

What it led to: the compute-optimal training arguments in Chinchilla, and later Gemini-family reports. Live notes: [2203-15556-training-compute-optimal-llms](https://iinventedlight.com/library/notes/2203-15556-training-compute-optimal-llms/). This week's batch also has [2312-11805-gemini](https://iinventedlight.com/library/notes/2312-11805-gemini/).

Use it when you want the original Gopher scaling study, not only the later compute-optimal follow-up. Where to read it: https://arxiv.org/abs/2112.11446 (v2 (21 Jan 2022)).

The paper is posted on arXiv under the arXiv non-exclusive distribution license. This note links to it rather than reproducing it. The note itself is CC BY 4.0.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).
