← All notes

Summary

Training Verifiers to Solve Math Word Problems (2021), by Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, and coauthors, is at https://arxiv.org/abs/2110.14168. GSM8K is described as a dataset of 8.5K high quality, linguistically diverse grade school math word problems, built so researchers can study multi-step quantitative reasoning. They find that even the largest transformer models fail to achieve high test performance, despite the conceptual simplicity of this problem distribution. They propose training verifiers to judge the correctness of model completions. At test time, the system generates many candidate solutions and keeps the one the verifier ranks highest. They report that verification helped on GSM8K and that it scaled more effectively with added data than a finetuning baseline. At test time, they generate many candidate solutions and select the one ranked highest by the verifier. They demonstrate that verification significantly improves performance on GSM8K, and they provide empirical evidence that verification scales more effectively with increased data than a finetuning baseline. Title, authors, abstract, and comments were taken from the arXiv abs page. The version is v2 (18 Nov 2021), submitted 27 Oct 2021.

Claims

Pointer notes do not quote the paper. See Verification and Sources for the link and license.

Verification

• Format: pointer note only. The paper is not reproduced here.
• Why it matters: It matters for two reasons that still shape math-reasoning work: it introduced GSM8K, and it argued for training a verifier rather than only finetuning a generator.
• Reproduced: not reproduced (2026-11-11).
• Related work: Related notes already in the library include Chain-of-Thought Prompting and Self-Consistency.
• Limits: This is a pointer note only. The paper is the original GSM8K release and verifier study from late 2021; later model scores are outside its scope.
• License: arXiv default (non-exclusive distribution license) on the abs page (2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/). Submitted 27 Oct 2021 (v1); last revised 18 Nov 2021 (v2). Sources: [S1] https://arxiv.org/abs/2110.14168, v2 (18 Nov 2021). checked_on: 2026-11-11. Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it. The note itself is CC BY 4.0.

When to apply

Use when you need the original GSM8K paper and the idea of training a verifier over many candidate solutions.

Facts

• id: F1 claim: 'The paper introduces GSM8K, a dataset of 8.5K grade school math word problems.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F2 claim: 'The authors propose training verifiers to judge the correctness of model completions.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F3 claim: 'At test time they generate many candidate solutions and select the one ranked highest by the verifier.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0

Sources

• [S1] arXiv 2110.14168, https://arxiv.org/abs/2110.14168. Paper license: arXiv default (non-exclusive distribution license). v2 (18 Nov 2021). Accessed 2026-11-11. This note links to the paper and does not reproduce it.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

@misc{iil_note_ej891kry,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{Training Verifiers to Solve Math Word Problems}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-11},
  url = {https://iinventedlight.com/library/notes/2110-14168-gsm8k/},
  note = {Permanent ID iil:note:ej891kry. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2110-14168-gsm8k.md.