FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI
Summary
FrontierMath is a benchmark of hundreds of original, exceptionally challenging mathematics problems crafted and vetted by expert mathematicians, covering most major branches of modern mathematics. The authors say that solving a typical problem requires multiple hours of effort from a researcher in the relevant branch, and that upper-end questions can take multiple days. The benchmark uses new, unpublished problems and automated verification to evaluate models while minimizing the risk of data contamination. They report that current state-of-the-art AI models solve under 2% of problems. They also release representative samples from the benchmark. [S1]
Claims
- id: C1 claim: 'FrontierMath is a benchmark of hundreds of original, exceptionally challenging mathematics problems.' source: S1 evidence: "a benchmark of hundreds of original, exceptionally challenging mathematics problems"
- id: C2 claim: 'The authors report that current state-of-the-art AI models solve under 2% of problems.' source: S1 evidence: "Current state-of-the-art AI models solve under 2% of problems"
- id: C3 claim: 'The benchmark uses new, unpublished problems and automated verification.' source: S1 evidence: "uses new, unpublished problems and automated verification"
Verification
- Code available: No official benchmark-host URL was named on the abs page. https://epochai.org returned HTTP 200 and redirected to https://epoch.ai/ on 2026-11-12. Not run.
- Reproduced: not reproduced (2026-11-12). Code not run.
- Related work: Live notes 2311-12022-gpqa and 2501-14249-humanitys-last-exam.
- Limits: The under-2% figure is from the v7 abstract last revised 23 Dec 2025 and is author-reported. Most problems are not public; the authors release representative samples.
- License: CC BY 4.0 on the abs page (2026-11-12; icon by-4.0.png; http://creativecommons.org/licenses/by/4.0/). Submitted 7 Nov 2024 (v1); last revised 23 Dec 2025 (v7). Sources: [S1] https://arxiv.org/abs/2411.04872, v7 (23 Dec 2025). checked_on: 2026-11-12.
When to apply
Use when you need a hard, expert-written math benchmark designed to reduce contamination.
Facts
- id: F1 claim: 'FrontierMath consists of hundreds of original, expert-vetted mathematics problems across major branches of modern mathematics.' confidence: medium date: 2026-11-12 sources: [S1] license: CC BY 4.0
- id: F2 claim: 'The authors report that current state-of-the-art AI models solve under 2% of the problems.' confidence: medium date: 2026-11-12 sources: [S1] license: CC BY 4.0
- id: F3 claim: 'Problems are new and unpublished, with automated verification, to reduce data-contamination risk.' confidence: medium date: 2026-11-12 sources: [S1] license: CC BY 4.0
Sources
- [S1] arXiv 2411.04872, https://arxiv.org/abs/2411.04872. License: CC BY 4.0. v7 (23 Dec 2025). Accessed 2026-11-12.
License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.
License: CC-BY-4.0 · Summary by iinventedlight (AI-assisted). Source: Elliot Glazer, Ege Erdil, Tamay Besiroglu, and 21 coauthors (listed on the arXiv abs page), FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI (CC BY 4.0).
Cite this
'iinventedlight (AI-assisted)'. 2026. "FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI." iinventedlight Library, iil:note:6xb81ycy, version of 2026-11-12. https://iinventedlight.com/library/notes/2411-04872-frontiermath/. License: CC-BY-4.0.
@misc{iil_note_6xb81ycy,
author = {{'iinventedlight (AI-assisted)'}},
title = {{FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI}},
howpublished = {iinventedlight Library},
year = {2026},
date = {2026-11-12},
url = {https://iinventedlight.com/library/notes/2411-04872-frontiermath/},
note = {Permanent ID iil:note:6xb81ycy. License: CC-BY-4.0.},
}
Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.
For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2411-04872-frontiermath.md.