← All notes

Summary

Let's Verify Step by Step (2023), by Hunter Lightman, Vineet Kosaraju, Yura Burda, and colleagues, is at the arXiv abs page. The paper is at https://arxiv.org/abs/2305.20050. They find that process supervision significantly outperforms outcome supervision for training models to solve problems from the challenging MATH dataset. In their account, a process-supervised model solved 78% of problems from a representative subset of the MATH test set, and active learning improved the value of process supervision. To support later work they released PRM800K, described as the complete dataset of 800,000 step-level human feedback labels used to train their best reward model. Even state of the art models still regularly produce logical mistakes. Outcome supervision provides feedback for a final result, and process supervision provides feedback for each intermediate reasoning step. They also show that active learning significantly improves the efficacy of process supervision. Title, authors, abstract, and comments were taken from the arXiv abs page. The version is v1 (31 May 2023), submitted 31 May 2023.

Claims

Pointer notes do not quote the paper. See Verification and Sources for the link and license.

Verification

• Format: pointer note only. The paper is not reproduced here.
• Why it matters: It matters because it gives a careful comparison of two supervision styles: outcome supervision, which scores only the final answer, and process supervision, which scores each intermediate reasoning step.
• Reproduced: not reproduced (2026-11-11).
• Related work: Related notes already in the library include Chain-of-Thought Prompting and Self-Consistency.
• Limits: This is a pointer note only. Reported figures are those named by the authors in the 31 May 2023 version and are not re-checked here.
• License: arXiv default (non-exclusive distribution license) on the abs page (2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/). Submitted 31 May 2023 (v1). Sources: [S1] https://arxiv.org/abs/2305.20050, v1 (31 May 2023). checked_on: 2026-11-11. Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it. The note itself is CC BY 4.0.

When to apply

Use when you need the original paper on process supervision versus outcome supervision for math reasoning, including the PRM800K release.

Facts

• id: F1 claim: 'The 2023 paper Let''s Verify Step by Step compares process supervision with outcome supervision for training models on MATH.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F2 claim: 'The authors report that a process-supervised model solved 78% of problems from a representative subset of the MATH test set.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F3 claim: 'They released PRM800K, a dataset of 800,000 step-level human feedback labels.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0

Sources

• [S1] arXiv 2305.20050, https://arxiv.org/abs/2305.20050. Paper license: arXiv default (non-exclusive distribution license). v1 (31 May 2023). Accessed 2026-11-11. This note links to the paper and does not reproduce it.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

@misc{iil_note_vk9nvxae,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{Let's Verify Step by Step}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-11},
  url = {https://iinventedlight.com/library/notes/2305-20050-lets-verify-step-by-step/},
  note = {Permanent ID iil:note:vk9nvxae. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2305-20050-lets-verify-step-by-step.md.