--- title: 'Training Verifiers to Solve Math Word Problems' license: CC-BY-4.0 license_url: https://creativecommons.org/licenses/by/4.0/ license_note: 'This note is CC BY 4.0. It links to the paper and does not reproduce it. Paper license: arXiv non-exclusive distribution (verified on arXiv abs page 2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/).' source: https://iinventedlight.com/library/notes/2110-14168-gsm8k.md source_license: 'arXiv default (non-exclusive distribution license)' attribution: 'Pointer note by iinventedlight (AI-assisted). Links to: Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, et al. (12 authors), Training Verifiers to Solve Math Word Problems (arXiv). This note does not reproduce the paper.' author: 'iinventedlight (AI-assisted)' date: 2026-11-11 status: published approved_by: "Cole Withington, 2026-11-11 20:58 PT, Grok Bot chat widget t1717s7407: \"Publish all 100 including the LLMs Encode Clinical Knowledge note\"" checked_on: 2026-11-11 as_of: 2026-11-11 confidence: medium untested: 'Pointer note only. Abs page read 2026-11-11. The paper is not reproduced here.' category: arxiv topic_area: 'ai-research: reasoning' tags: arxiv, pointer, reasoning arxiv_id: 2110.14168 alias: 2110.14168-gsm8k format: pointer type: pointer sources: - id: S1 title: 'Training Verifiers to Solve Math Word Problems' publisher: arXiv authors: 'Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, et al. (12 authors)' url: https://arxiv.org/abs/2110.14168 version_read: 'v2 (18 Nov 2021)' accessed: 2026-11-11 license: 'arXiv default (non-exclusive distribution license)' --- # Training Verifiers to Solve Math Word Problems ## Summary Training Verifiers to Solve Math Word Problems (2021), by Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, and coauthors, is at https://arxiv.org/abs/2110.14168. GSM8K is described as a dataset of 8.5K high quality, linguistically diverse grade school math word problems, built so researchers can study multi-step quantitative reasoning. They find that even the largest transformer models fail to achieve high test performance, despite the conceptual simplicity of this problem distribution. They propose training verifiers to judge the correctness of model completions. At test time, the system generates many candidate solutions and keeps the one the verifier ranks highest. They report that verification helped on GSM8K and that it scaled more effectively with added data than a finetuning baseline. At test time, they generate many candidate solutions and select the one ranked highest by the verifier. They demonstrate that verification significantly improves performance on GSM8K, and they provide empirical evidence that verification scales more effectively with increased data than a finetuning baseline. Title, authors, abstract, and comments were taken from the arXiv abs page. The version is v2 (18 Nov 2021), submitted 27 Oct 2021. ## Claims Pointer notes do not quote the paper. See Verification and Sources for the link and license. ## Verification - Format: pointer note only. The paper is not reproduced here. - Why it matters: It matters for two reasons that still shape math-reasoning work: it introduced GSM8K, and it argued for training a verifier rather than only finetuning a generator. - Reproduced: not reproduced (2026-11-11). - Related work: Related notes already in the library include [Chain-of-Thought Prompting](https://iinventedlight.com/library/notes/2201-11903-chain-of-thought-prompting/) and [Self-Consistency](https://iinventedlight.com/library/notes/2203-11171-self-consistency/). - Limits: This is a pointer note only. The paper is the original GSM8K release and verifier study from late 2021; later model scores are outside its scope. - License: arXiv default (non-exclusive distribution license) on the abs page (2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/). Submitted 27 Oct 2021 (v1); last revised 18 Nov 2021 (v2). Sources: [S1] https://arxiv.org/abs/2110.14168, v2 (18 Nov 2021). checked_on: 2026-11-11. Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it. The note itself is CC BY 4.0. ## When to apply Use when you need the original GSM8K paper and the idea of training a verifier over many candidate solutions. ## Facts - id: F1 claim: 'The paper introduces GSM8K, a dataset of 8.5K grade school math word problems.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0 - id: F2 claim: 'The authors propose training verifiers to judge the correctness of model completions.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0 - id: F3 claim: 'At test time they generate many candidate solutions and select the one ranked highest by the verifier.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0 ## Sources - [S1] arXiv 2110.14168, https://arxiv.org/abs/2110.14168. Paper license: arXiv default (non-exclusive distribution license). v2 (18 Nov 2021). Accessed 2026-11-11. This note links to the paper and does not reproduce it. License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it. --- Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).