Language Models are Few-Shot Learners
Summary
Language Models are Few-Shot Learners (2020) is the GPT-3 paper. It argued that as language models get large enough, they can pick up a new task from a short prompt with a few examples, without a separate fine-tuning run for that task. That in-context, few-shot result is what turned large language models into a general interface, not just a better pretrained classifier.
What it introduced: a 175-billion-parameter autoregressive language model, a systematic study of zero-, one-, and few-shot prompting, and a public account of how far next-token pretraining could go at that scale.
What it led to: instruction-following assistants, tool-using agents, and the later scaling and alignment papers in this batch (InstructGPT, the GPT-4 technical report, Llama 2 and Llama 3). Live notes on later preference-training methods include 2204-05862-helpful-harmless-rlhf and 2305-18290-direct-preference-optimization.
Where to read it: https://arxiv.org/abs/2005.14165 (v4, last revised 22 Jul 2020).
The paper is posted on arXiv under the arXiv non-exclusive distribution license. This note links to it rather than reproducing it. The note itself is CC BY 4.0.
License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.
License: CC-BY-4.0 · Pointer by iinventedlight (AI-assisted). Source: Tom B. Brown and 30 coauthors (listed on the arXiv abs page), Language Models are Few-Shot Learners (arXiv non-exclusive distribution license). This note links to the paper rather than reproducing it.
Cite this
'iinventedlight (AI-assisted)'. 2026. "Language Models are Few-Shot Learners." iinventedlight Library, iil:note:szp0amzd, version of 2026-11-11. https://iinventedlight.com/library/notes/2005-14165-language-models-are-few-shot-learners/. License: CC-BY-4.0.
@misc{iil_note_szp0amzd,
author = {{'iinventedlight (AI-assisted)'}},
title = {{Language Models are Few-Shot Learners}},
howpublished = {iinventedlight Library},
year = {2026},
date = {2026-11-11},
url = {https://iinventedlight.com/library/notes/2005-14165-language-models-are-few-shot-learners/},
note = {Permanent ID iil:note:szp0amzd. License: CC-BY-4.0.},
}Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.
For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2005-14165-language-models-are-few-shot-learners.md.