← All notes

Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement

Summary

Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement (2024) describes the Qwen2.5-Math and Qwen2.5-Math-Instruct models at 1.5B, 7B, and 72B, trained with a self-improvement loop rather than a single supervised pass.

What it introduced: self-improvement across pretraining, post-training, and inference. A prior math model generates pretraining data; a reward model is built from massive sampling and is used to iterate supervised data; that stronger model then updates the reward model; reinforcement learning is applied at the end; and the same reward model can guide sampling at inference. The instruction models support Chinese and English and can use chain-of-thought or tool-integrated reasoning. Evaluation covers ten math datasets, from grade-school sets such as GSM8K to contest sets such as AIME24.

What it led to: a strong open math line sitting next to DeepSeekMath and later Qwen3 reasoning models. Live notes: 2402-03300-deepseekmath and 2110-14168-gsm8k. The Qwen2.5 and Qwen3 reports are pointer notes in this week's batch F.

Where to read it: https://arxiv.org/abs/2409.12122 (v1, 18 Sep 2024).

The paper is posted on arXiv under the arXiv non-exclusive distribution license. This note links to it rather than reproducing it. The note itself is CC BY 4.0.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

License: CC-BY-4.0 · Pointer by iinventedlight (AI-assisted). Source: An Yang, Beichen Zhang, Binyuan Hui, and 13 coauthors, Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement (arXiv non-exclusive distribution license). This note links to the paper rather than reproducing it.

Cite this

Permanent ID: iil:note:88enb2v4

'iinventedlight (AI-assisted)'. 2026. "Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement." iinventedlight Library, iil:note:88enb2v4, version of 2026-11-12. https://iinventedlight.com/library/notes/2409-12122-qwen2.5-math/. License: CC-BY-4.0.

@misc{iil_note_88enb2v4,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-12},
  url = {https://iinventedlight.com/library/notes/2409-12122-qwen2.5-math/},
  note = {Permanent ID iil:note:88enb2v4. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2409-12122-qwen2.5-math.md.