← All notes

Short answer

Finetuning a pretrained language model on many tasks written as instructions makes it better at new, unseen tasks. This paper scales that recipe along three axes: more tasks, bigger models, and chain-of-thought data. [S1]

What they did

• Studied instruction finetuning while increasing the number of tasks, the model size, and adding chain-of-thought (step-by-step) examples. [S1]
• Applied it to several model families (PaLM, T5, U-PaLM) and tested zero-shot, few-shot, and chain-of-thought prompting. [S1]
• Flan-PaLM 540B, finetuned on about 1.8K tasks, beat the base PaLM 540B by 9.4% on average. [S1]
• Released Flan-T5 checkpoints, which perform well in few-shot settings even against much larger models. [S1]

Why it matters

The authors present instruction finetuning as a general way to make pretrained models both stronger and easier to use. [S1]

Limits

This note covers only the abstract. Benchmark numbers are as reported by the authors and were not re-run.

Claims

• id: C1 claim: The paper scales instruction finetuning by number of tasks, model size, and chain-of-thought data. source: S1 evidence: "scaling the number of tasks, (2) scaling the model size"
• id: C2 claim: Flan-PaLM 540B finetuned on 1.8K tasks outperforms PaLM 540B by 9.4% on average. source: S1 evidence: "outperforms PALM 540B by a large margin (+9.4% on average)"
• id: C3 claim: The authors released Flan-T5 checkpoints. source: S1 evidence: "We also publicly release Flan-T5 checkpoints"
• id: C4 claim: The authors call instruction finetuning a general method for improving pretrained models. source: S1 evidence: "instruction finetuning is a general method for improving the performance"

Sources

• [S1] Scaling Instruction-Finetuned Language Models. arXiv. https://arxiv.org/abs/2210.11416 . License: CC BY 4.0. Accessed 2026-11-05.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).

@misc{iil_note_rvzc071p,
  author = {{iinventedlight}},
  title = {{Scaling Instruction-Finetuned Language Models (Flan)}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-04},
  url = {https://iinventedlight.com/library/notes/2210-11416-flan-instruction-finetuning/},
  note = {Permanent ID iil:note:rvzc071p. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/2210-11416-flan-instruction-finetuning.md.