MATH
Introduced by Dan Hendrycks et al. in
Measuring Mathematical Problem Solving With the MATH Dataset 5 Mar 2021 archive 2025-07-28
MATH is a new dataset of 12,500 challenging competition mathematics problems. Each problem in MATH has a full step-by-step solution which can be used to teach models to generate answer derivations and explanations.
Source: Hendrycks et al.
Image source: Hendrycks et al.
Benchmarks archive 2025-07-28
All 2 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.
Syntology 54 newer papers were judged to report on this dataset's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 0 placed): Math Word Problem Solving · MATH (53 judged, 0 placed), Math Word Problem Solving · MATH minival (1 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).
Papers archive 2025-07-28
papers with a benchmark row: 37 · with a code link: 34 · where Syntology ran a sample: 21 (17 with a run with no instrument failure, 4 where every run was a failure of Syntology's instrument) Syntology
Show:
all papers with a benchmark row only where code ran (21 of 37 with a benchmark row: 17 with a run with no instrument failure, 4 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 37 of 37 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).
The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 1,330. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed 2026-10-05 ; when this build read Syntology's graph is in the build record .
Paper Code Results Date Samples run Syntology
OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data
1
4
2 Oct 2024
12 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 0 violated, 4 with no contract checked; 7 where Syntology's instrument failed) · 3 unverified
Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
0
6
18 Sep 2024
not harvested
Qwen2 Technical Report
6
1
15 Jul 2024
not harvested
Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs
1
1
26 Jun 2024
official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 0 violated, 4 with no contract checked; 6 where Syntology's instrument failed) · 1 unverified (12 pointer-only for licence )
DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving
1
8
18 Jun 2024
official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 8 where Syntology's instrument failed) · 2 unverified
AlphaMath Almost Zero: Process Supervision without Process
1
1
6 May 2024
not harvested
Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing
1
1
18 Apr 2024
official (archive's flag): 14 ran · 14 ran (of which 1 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 1 violated, 3 with no contract checked; 9 where Syntology's instrument failed) · 4 unverified (18 pointer-only for licence )
MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problems
1
1
6 Apr 2024
official: harvested, nothing ran · 0 ran · 1 unverified (1 pointer-only for licence )
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
1
1
12 Mar 2024
not harvested
Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning
0
4
4 Mar 2024
not harvested
An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning
1
4
23 Feb 2024
official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 2 violated, 3 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified (11 pointer-only for licence )
OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset
1
12
15 Feb 2024
not harvested
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
5
2
5 Feb 2024
official (archive's flag): 8 ran · 14 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 11 where Syntology's instrument failed) · 10 unverified (3 pointer-only for licence )
Augmenting Math Word Problems via Iterative Question Composing
1
1
17 Jan 2024
official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (4 pointer-only for licence )
Mixtral of Experts
6
2
8 Jan 2024
5 ran (of which 5 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 5 samples that ran constructed an object rather than computing a result
Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
2
2
5 Jan 2024
not harvested
Gemini: A Family of Highly Capable Multimodal Models
1
2
19 Dec 2023
not harvested
Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
3
3
14 Dec 2023
not harvested
Mistral 7B
6
1
10 Oct 2023
official (archive's flag): 9 ran · 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (2 pointer-only for licence )
MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning
1
3
9 Oct 2023
not harvested
MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning
1
6
5 Oct 2023
official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified
ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving
1
8
29 Sep 2023
official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 2 violated, 3 with no contract checked; 6 where Syntology's instrument failed) · 1 unverified (9 pointer-only for licence )
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
1
3
21 Sep 2023
official (archive's flag): 15 ran · 15 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 14 where Syntology's instrument failed) · 7 unverified
OpenChat: Advancing Open-source Language Models with Mixed-Quality Data
1
2
20 Sep 2023
official: harvested, nothing ran · 0 ran · 1 unverified
WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct
1
4
18 Aug 2023
10 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 3 where Syntology's instrument failed) · 6 unverified (16 pointer-only for licence )
Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification
1
5
15 Aug 2023
not harvested
Cumulative Reasoning with Large Language Models
1
2
8 Aug 2023
official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (6 pointer-only for licence )
Skills-in-Context Prompting: Unlocking Compositionality in Large Language Models
0
1
1 Aug 2023
not harvested
Let's Verify Step by Step
3
1
31 May 2023
official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 2 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified
PaLM 2 Technical Report
1
2
17 May 2023
not harvested
Progressive-Hint Prompting Improves Reasoning in Large Language Models
1
1
19 Apr 2023
official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (4 pointer-only for licence )
Sparks of Artificial General Intelligence: Early experiments with GPT-4
3
1
22 Mar 2023
9 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified
LLaMA: Open and Efficient Foundation Language Models
57
8
27 Feb 2023
official: no sample here; runs from other or unrecorded repositories · 37 ran (of which 9 constructed an object rather than computing a result; 25 with no instrument failure: 3 honoured, 0 violated, 22 with no contract checked; 12 where Syntology's instrument failed) · 21 unverified (4 pointer-only for licence )
PAL: Program-aided Language Models
3
1
18 Nov 2022
3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified
Galactica: A Large Language Model for Science
1
7
16 Nov 2022
official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified
Solving Quantitative Reasoning Problems with Language Models
1
13
29 Jun 2022
not harvested
Measuring Mathematical Problem Solving With the MATH Dataset
5
8
5 Mar 2021
official (archive's flag): 2 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (1 pointer-only for licence )
Dataset loaders archive 2025-07-28
2 loaders as listed in the archive; links are outbound and not re-checked here.
Tasks archive 2025-07-28
License archive 2025-07-28
MIT
Modalities archive 2025-07-28
Languages archive 2025-07-28
Variants archive 2025-07-28
2 variant names, as the archive lists them.