← All notes

Summary

DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (2024), by Zhihong Shao, Peiyi Wang, Qihao Zhu, and coauthors, is at https://arxiv.org/abs/2402.03300. The authors present DeepSeekMath 7B, obtained by continued pre-training of DeepSeek-Coder-Base-v1.5 7B with 120B math-related tokens sourced from Common Crawl, together with natural language and code data. They report a score of 51.7% on the competition-level MATH benchmark without external toolkits and voting techniques, and 60.9% with self-consistency over 64 samples. They attribute the result to a data selection pipeline over public web data and to GRPO, which they describe as a variant of Proximal Policy Optimization that is meant to improve mathematical reasoning while using memory more carefully than PPO. Self-consistency over 64 samples from DeepSeekMath 7B achieves 60.9% on MATH. The authors attribute the mathematical reasoning capability to a data selection pipeline over publicly available web data, and to Group Relative Policy Optimization, a variant of Proximal Policy Optimization that enhances mathematical reasoning abilities while concurrently optimizing the memory usage of PPO. Title, authors, abstract, and comments were taken from the arXiv abs page. The version is v3 (27 Apr 2024), submitted 5 Feb 2024.

Claims

Pointer notes do not quote the paper. See Verification and Sources for the link and license.

Verification

• Format: pointer note only. The paper is not reproduced here.
• Why it matters: It matters as a widely cited open-model result on competition-style math, and as the paper that introduces Group Relative Policy Optimization, or GRPO, later used in other open reasoning models.
• Reproduced: not reproduced (2026-11-11).
• Related work: Related notes already in the library include Chain-of-Thought Prompting and Self-Consistency.
• Limits: This is a pointer note only. MATH scores are the authors' reported results from the v3 paper (27 Apr 2024).
• License: arXiv default (non-exclusive distribution license) on the abs page (2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/). Submitted 5 Feb 2024 (v1); last revised 27 Apr 2024 (v3). Sources: [S1] https://arxiv.org/abs/2402.03300, v3 (27 Apr 2024). checked_on: 2026-11-11. Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it. The note itself is CC BY 4.0.

When to apply

Use when tracing open-model math reasoning, continued pre-training on math text, or the GRPO method named in this paper.

Facts

• id: F1 claim: 'DeepSeekMath 7B continues pre-training DeepSeek-Coder-Base-v1.5 7B with 120B math-related tokens from Common Crawl, plus natural language and code data.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F2 claim: 'The authors report 51.7% on the competition-level MATH benchmark without external toolkits and voting, and 60.9% with self-consistency over 64 samples.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F3 claim: 'They introduce Group Relative Policy Optimization (GRPO), described as a variant of Proximal Policy Optimization.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0

Sources

• [S1] arXiv 2402.03300, https://arxiv.org/abs/2402.03300. Paper license: arXiv default (non-exclusive distribution license). v3 (27 Apr 2024). Accessed 2026-11-11. This note links to the paper and does not reproduce it.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

@misc{iil_note_7dj2g57d,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-11},
  url = {https://iinventedlight.com/library/notes/2402-03300-deepseekmath/},
  note = {Permanent ID iil:note:7dj2g57d. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2402-03300-deepseekmath.md.