Textbooks Are All You Need
Summary
This paper introduces phi-1, a Transformer-based code model with 1.3B parameters, trained for 4 days on 8 A100s. Training uses a selection of textbook-quality data from the web (6B tokens) and synthetically generated textbooks and exercises with GPT-3.5 (1B tokens). The authors report that phi-1 attains pass@1 accuracy of 50.6% on HumanEval and 55.5% on MBPP. They also compare phi-1-base, the model before a finetuning stage on coding exercises, and phi-1-small, a 350M-parameter model trained with the same pipeline, which they say still achieves 45% on HumanEval. [S1]
Claims
- id: C1 claim: 'phi-1 is a 1.3B-parameter Transformer trained for 4 days on 8 A100s.' source: S1 evidence: "phi-1 is a Transformer-based model with 1.3B parameters, trained for 4 days on 8 A100s"
- id: C2 claim: 'Training used 6B tokens of textbook-quality web data and 1B tokens of GPT-3.5 synthetic textbooks and exercises.' source: S1 evidence: "synthetically generated textbooks and exercises with GPT-3.5 (1B tokens)"
- id: C3 claim: 'The authors report 50.6% pass@1 on HumanEval and 55.5% on MBPP.' source: S1 evidence: "pass@1 accuracy 50.6% on HumanEval and 55.5% on MBPP"
Verification
- Code available: The authors say they release the model for usage and evaluation, but no official model URL was named on the abs page or HTML.
- Reproduced: not reproduced (2026-11-12). Code not run.
- Related work: This batch also has the later Phi-3 technical report and the Code Llama note.
- Limits: The 50.6% and 55.5% scores are author-reported (v2, 2 Oct 2023). The authors omit some synthetic-data details and discuss possible HumanEval contamination.
- License: CC BY 4.0 on the abs page (2026-11-12; icon by-4.0.png; http://creativecommons.org/licenses/by/4.0/). Submitted 20 Jun 2023 (v1); last revised 2 Oct 2023 (v2). Sources: [S1] https://arxiv.org/abs/2306.11644, v2 (2 Oct 2023). checked_on: 2026-11-12.
When to apply
Use when asking how textbook-quality and synthetic data were used to train the small phi-1 code model.
Facts
- id: F1 claim: 'phi-1 is a 1.3B-parameter code model trained for 4 days on 8 A100s.' confidence: medium date: 2026-11-12 sources: [S1] license: CC BY 4.0
- id: F2 claim: 'The authors use 6B tokens of textbook-quality web data and 1B tokens of GPT-3.5 synthetic textbooks and exercises.' confidence: medium date: 2026-11-12 sources: [S1] license: CC BY 4.0
- id: F3 claim: 'They report 50.6% pass@1 on HumanEval and 55.5% on MBPP, and 45% HumanEval for a 350M phi-1-small model.' confidence: medium date: 2026-11-12 sources: [S1] license: CC BY 4.0
Sources
- [S1] arXiv 2306.11644, https://arxiv.org/abs/2306.11644. License: CC BY 4.0. v2 (2 Oct 2023). Accessed 2026-11-12.
License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.
License: CC-BY-4.0 · Summary by iinventedlight (AI-assisted). Source: Suriya Gunasekar, Yi Zhang, Jyoti Aneja, and 16 coauthors (listed on the arXiv abs page), Textbooks Are All You Need (CC BY 4.0).
Cite this
'iinventedlight (AI-assisted)'. 2026. "Textbooks Are All You Need." iinventedlight Library, iil:note:0t15b8va, version of 2026-11-12. https://iinventedlight.com/library/notes/2306-11644-textbooks-are-all-you-need/. License: CC-BY-4.0.
@misc{iil_note_0t15b8va,
author = {{'iinventedlight (AI-assisted)'}},
title = {{Textbooks Are All You Need}},
howpublished = {iinventedlight Library},
year = {2026},
date = {2026-11-12},
url = {https://iinventedlight.com/library/notes/2306-11644-textbooks-are-all-you-need/},
note = {Permanent ID iil:note:0t15b8va. License: CC-BY-4.0.},
}
Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.
For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2306-11644-textbooks-are-all-you-need.md.