Math Word Problem Solving

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

13 leaderboard tables shown for this task, 13 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 13 until expanded.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

23 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 80 papers with code (107 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• LLaMA: Open and Efficient Foundation Language Models
  27 Feb 2023 57 repositories listed Syntology ran 26 of 58 samples · 32 unverified · 4 pointer-only (licence)
  We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters.
• Llama 2: Open Foundation and Fine-Tuned Chat Models
  18 Jul 2023 19 repositories listed Syntology ran 31 of 52 samples · 21 unverified · 16 pointer-only (licence)
  In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters.
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention
  5 Jun 2020 14 repositories listed Syntology ran 4 of 13 samples · 9 unverified · 3 pointer-only (licence)
  Recent progress in pre-trained neural language models has significantly improved the performance of many natural language processing (NLP) tasks.
• Analysing Mathematical Reasoning Abilities of Neural Models
  2 Apr 2019 7 repositories listed Syntology ran 0 of 1 samples · 1 unverified
  The structured nature of the mathematics domain, covering arithmetic, algebra, probability and calculus, enables the construction of training and test splits designed to clearly illuminate the capabilities and…
• Qwen2 Technical Report
  15 Jul 2024 6 repositories listed
  This report introduces the Qwen2 series, the latest addition to our large language models and large multimodal models.
• Mixtral of Experts
  8 Jan 2024 6 repositories listed Syntology ran 5 of 5 samples · 0 unverified
  In particular, Mixtral vastly outperforms Llama 2 70B on mathematics, code generation, and multilingual benchmarks.
• Mistral 7B
  10 Oct 2023 6 repositories listed Syntology ran 9 of 11 samples · 2 unverified · 1 pointer-only (licence)
  We introduce Mistral 7B v0.
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
  5 Feb 2024 5 repositories listed Syntology ran 8 of 24 samples · 16 unverified
  Mathematical reasoning poses a significant challenge for language models due to its complex and structured nature.
• Measuring Mathematical Problem Solving With the MATH Dataset
  5 Mar 2021 5 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 1 pointer-only (licence)
  To facilitate future research and increase accuracy on MATH, we also contribute a large auxiliary pretraining dataset which helps teach models the fundamentals of mathematics.
• Large Language Models are Zero-Shot Reasoners
  24 May 2022 4 repositories listed Syntology ran 0 of 4 samples · 4 unverified · 1 pointer-only (licence)
  Pretrained large language models (LLMs) are widely used in many sub-fields of natural language processing (NLP) and generally known as excellent few-shot learners with task-specific exemplars.
• Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
  14 Dec 2023 3 repositories listed
  In this paper, we present an innovative process-oriented math process reward model called \textbf{Math-Shepherd}, which assigns a reward score to each step of math problem solutions.
• Let's Verify Step by Step
  31 May 2023 3 repositories listed Syntology ran 3 of 3 samples · 0 unverified
  We conduct our own investigation, finding that process supervision significantly outperforms outcome supervision for training models to solve problems from the challenging MATH dataset.
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
  22 Mar 2023 3 repositories listed Syntology ran 0 of 10 samples · 10 unverified
  We contend that (this early version of) GPT-4 is part of a new cohort of LLMs (along with ChatGPT and Google's PaLM for example) that exhibit more general intelligence than previous AI models.
• PAL: Program-aided Language Models
  18 Nov 2022 3 repositories listed Syntology ran 3 of 3 samples · 0 unverified
  Much of this success can be attributed to prompting methods such as "chain-of-thought'', which employ LLMs for both understanding the problem description by decomposing it into steps, as well as solving each step of the…
• Are NLP Models really able to Solve Simple Math Word Problems?
  12 Mar 2021 3 repositories listed
  Since existing solvers achieve high performance on the benchmark datasets for elementary level MWPs containing one-unknown arithmetic word problems, such problems are often considered "solved" with the bulk of research…
• Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
  5 Jan 2024 2 repositories listed
  Using PESC during instruction tuning, our best sparse model outperforms other sparse and dense models and exhibits superior general capabilities compared to GPT-3.
• MathChat: Converse to Tackle Challenging Math Problems with LLM Agents
  2 Jun 2023 2 repositories listed
  Employing Large Language Models (LLMs) to address mathematical problems is an intriguing research endeavor, considering the abundance of math problems expressed in natural language across numerous science and…
• RetICL: Sequential Retrieval of In-Context Examples with Reinforcement Learning
  23 May 2023 2 repositories listed
  Recent developments in large pre-trained language models have enabled unprecedented performance on a variety of downstream tasks.
• LogicSolver: Towards Interpretable Math Word Problem Solving with Logical Prompt-enhanced Learning
  17 May 2022 2 repositories listed
  To address this issue and make a step towards interpretable MWP solving, we first construct a high-quality MWP dataset named InterMWP which consists of 11, 495 MWPs and annotates interpretable logical formulas based on…
• A Diversity-Enhanced Knowledge Distillation Model for Practical Math Word Problem Solving
  7 Jan 2025 1 repository listed
  Our approach proposes an adaptive diversity distillation method, in which a student model learns diverse equations by selectively transferring high-quality knowledge from a teacher model.
• SBI-RAG: Enhancing Math Word Problem Solving for Students through Schema-Based Instruction and Retrieval-Augmented Generation
  17 Oct 2024 1 repository listed
  Many students struggle with math word problems (MWPs), often finding it difficult to identify key information and select the appropriate mathematical operations.
• Teaching-Inspired Integrated Prompting Framework: A Novel Approach for Enhancing Reasoning in Large Language Models
  10 Oct 2024 1 repository listed Syntology ran 8 of 8 samples · 0 unverified · 8 pointer-only (licence)
  Experiments are conducted on nine benchmarks which demonstrates that our approach improves the reasoning accuracy of LLMs.
• OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data
  2 Oct 2024 1 repository listed Syntology ran 0 of 15 samples · 15 unverified
  However, most of the cutting-edge progress in mathematical reasoning with LLMs has become \emph{closed-source} due to lack of access to training data.
• Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs
  26 Jun 2024 1 repository listed Syntology ran 7 of 12 samples · 5 unverified · 12 pointer-only (licence)
  Mathematical reasoning presents a significant challenge for Large Language Models (LLMs) due to the extensive and precise chain of reasoning required for accuracy.
• DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving
  18 Jun 2024 1 repository listed Syntology ran 10 of 12 samples · 2 unverified
  Solving mathematical problems requires advanced reasoning abilities and presents notable challenges for large language models.
• AlphaMath Almost Zero: Process Supervision without Process
  6 May 2024 1 repository listed
  Although recent advancements in large language models (LLMs) have significantly improved their performance on various tasks, they still face challenges with complex and symbolic multi-step reasoning, particularly in…
• Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problems
  23 Apr 2024 1 repository listed
  To this end, we propose a simple-yet-effective method, namely Deeply Understanding the Problems (DUP), to improve the LLMs' math problem-solving ability by addressing semantic misunderstanding errors.
• Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing
  18 Apr 2024 1 repository listed Syntology ran 13 of 18 samples · 5 unverified · 18 pointer-only (licence)
  Despite the impressive capabilities of Large Language Models (LLMs) on various tasks, they still struggle with scenarios that involves complex reasoning and planning.
• MACM: Utilizing a Multi-Agent System for Condition Mining in Solving Complex Mathematical Problems
  6 Apr 2024 1 repository listed Syntology ran 0 of 1 samples · 1 unverified · 1 pointer-only (licence)
  Firstly, their effectiveness in tackling complex mathematical problems is somewhat constrained.
• Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
  12 Mar 2024 1 repository listed
  We investigate efficient methods for training Large Language Models (LLMs) to possess capabilities in multiple specialized domains, such as coding, math reasoning and world knowledge.

Syntology lines on 18 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.