--- title: 'DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models' license: CC-BY-4.0 license_url: https://creativecommons.org/licenses/by/4.0/ license_note: 'This note is CC BY 4.0. It links to the paper and does not reproduce it. Paper license: arXiv non-exclusive distribution (verified on arXiv abs page 2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/).' source: https://iinventedlight.com/library/notes/2402-03300-deepseekmath.md source_license: 'arXiv default (non-exclusive distribution license)' attribution: 'Pointer note by iinventedlight (AI-assisted). Links to: Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, et al. (11 authors), DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (arXiv). This note does not reproduce the paper.' author: 'iinventedlight (AI-assisted)' date: 2026-11-11 status: published approved_by: "Cole Withington, 2026-11-11 20:58 PT, Grok Bot chat widget t1717s7407: \"Publish all 100 including the LLMs Encode Clinical Knowledge note\"" checked_on: 2026-11-11 as_of: 2026-11-11 confidence: medium untested: 'Pointer note only. Abs page read 2026-11-11. The paper is not reproduced here.' category: arxiv topic_area: 'ai-research: reasoning' tags: arxiv, pointer, reasoning arxiv_id: 2402.03300 alias: 2402.03300-deepseekmath format: pointer type: pointer sources: - id: S1 title: 'DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models' publisher: arXiv authors: 'Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, et al. (11 authors)' url: https://arxiv.org/abs/2402.03300 version_read: 'v3 (27 Apr 2024)' accessed: 2026-11-11 license: 'arXiv default (non-exclusive distribution license)' --- # DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models ## Summary DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (2024), by Zhihong Shao, Peiyi Wang, Qihao Zhu, and coauthors, is at https://arxiv.org/abs/2402.03300. The authors present DeepSeekMath 7B, obtained by continued pre-training of DeepSeek-Coder-Base-v1.5 7B with 120B math-related tokens sourced from Common Crawl, together with natural language and code data. They report a score of 51.7% on the competition-level MATH benchmark without external toolkits and voting techniques, and 60.9% with self-consistency over 64 samples. They attribute the result to a data selection pipeline over public web data and to GRPO, which they describe as a variant of Proximal Policy Optimization that is meant to improve mathematical reasoning while using memory more carefully than PPO. Self-consistency over 64 samples from DeepSeekMath 7B achieves 60.9% on MATH. The authors attribute the mathematical reasoning capability to a data selection pipeline over publicly available web data, and to Group Relative Policy Optimization, a variant of Proximal Policy Optimization that enhances mathematical reasoning abilities while concurrently optimizing the memory usage of PPO. Title, authors, abstract, and comments were taken from the arXiv abs page. The version is v3 (27 Apr 2024), submitted 5 Feb 2024. ## Claims Pointer notes do not quote the paper. See Verification and Sources for the link and license. ## Verification - Format: pointer note only. The paper is not reproduced here. - Why it matters: It matters as a widely cited open-model result on competition-style math, and as the paper that introduces Group Relative Policy Optimization, or GRPO, later used in other open reasoning models. - Reproduced: not reproduced (2026-11-11). - Related work: Related notes already in the library include [Chain-of-Thought Prompting](https://iinventedlight.com/library/notes/2201-11903-chain-of-thought-prompting/) and [Self-Consistency](https://iinventedlight.com/library/notes/2203-11171-self-consistency/). - Limits: This is a pointer note only. MATH scores are the authors' reported results from the v3 paper (27 Apr 2024). - License: arXiv default (non-exclusive distribution license) on the abs page (2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/). Submitted 5 Feb 2024 (v1); last revised 27 Apr 2024 (v3). Sources: [S1] https://arxiv.org/abs/2402.03300, v3 (27 Apr 2024). checked_on: 2026-11-11. Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it. The note itself is CC BY 4.0. ## When to apply Use when tracing open-model math reasoning, continued pre-training on math text, or the GRPO method named in this paper. ## Facts - id: F1 claim: 'DeepSeekMath 7B continues pre-training DeepSeek-Coder-Base-v1.5 7B with 120B math-related tokens from Common Crawl, plus natural language and code data.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0 - id: F2 claim: 'The authors report 51.7% on the competition-level MATH benchmark without external toolkits and voting, and 60.9% with self-consistency over 64 samples.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0 - id: F3 claim: 'They introduce Group Relative Policy Optimization (GRPO), described as a variant of Proximal Policy Optimization.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0 ## Sources - [S1] arXiv 2402.03300, https://arxiv.org/abs/2402.03300. Paper license: arXiv default (non-exclusive distribution license). v3 (27 Apr 2024). Accessed 2026-11-11. This note links to the paper and does not reproduce it. License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it. --- Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).