Papers › MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning

MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning

9 Oct 2023arXiv:2310.05506archive 2025-07-28

Chengpeng Li, Zheng Yuan, Hongyi Yuan, Guanting Dong, Keming Lu, Jiancan Wu, Chuanqi Tan, Xiang Wang, Chang Zhou

In math reasoning with large language models (LLMs), fine-tuning data augmentation by query evolution and diverse reasoning paths is empirically verified effective, profoundly narrowing the gap between open-sourced LLMs and cutting-edge proprietary LLMs. In this paper, we conduct an investigation for such data augmentation in math reasoning and are intended to answer: (1) What strategies of data augmentation are more effective; (2) What is the scaling relationship between the amount of augmented data and model performance; and (3) Can data augmentation incentivize generalization to out-of-domain mathematical reasoning tasks? To this end, we create two new dataset AugGSM8K and AugMATH, by complicating and diversifying the queries and sampling multiple reasoning paths from GSM8K and MATH. We obtained a series of LLMs called MuggleMath by fine-tuning LLaMA models on AugGSM8K and AugMATH. MuggleMath substantially achieves new state-of-the-art on GSM8K and MATH. A log-linear relationship and a segmented log-linear are presented between MuggleMath's performance and the amount of augmented data on GSM8K and MATH, respectively. We also find that it is weak in out-of-domain math reasoning generalization from AugGSM8K to MATH and from AugMATH to GSM8K, which suggests that augmenting queries that cover a broader range of subjects is more beneficial for generalization. We release our codes and augmented data in https://github.com/OFA-Sys/gsm8k-ScRel.

PaperPDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

ofa-sys/gsm8k-screl officialmentioned in papermentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Scaling Relationship on Learning Mathematical Reasoning with Large Language Models (arXiv:2308.01825; that paper's own run record: 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Arithmetic ReasoningData AugmentationGSM8KMathMath Word Problem SolvingMathematical Reasoning

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Arithmetic Reasoning GSM8K MuggleMATH 70B Accuracy 82.3 #61 of 164 Archive leaderboard report
Arithmetic Reasoning GSM8K MuggleMATH 70B Parameters (Billion) 70 #61 of 164 Archive leaderboard report
Arithmetic Reasoning GSM8K MuggleMATH 13B Accuracy 74 #91 of 164 Archive leaderboard report
Arithmetic Reasoning GSM8K MuggleMATH 13B Parameters (Billion) 13 #91 of 164 Archive leaderboard report
Arithmetic Reasoning GSM8K MuggleMATH 7B Accuracy 69.8 #103 of 164 Archive leaderboard report
Arithmetic Reasoning GSM8K MuggleMATH 7B Parameters (Billion) 7 #103 of 164 Archive leaderboard report
Math Word Problem Solving MATH MuggleMATH-70B Accuracy 35.6 #80 of 135 Archive leaderboard report
Math Word Problem Solving MATH MuggleMATH-70B Parameters (Billions) 70 #80 of 135 Archive leaderboard report
Math Word Problem Solving MATH MuggleMATH-13B Accuracy 30.7 #87 of 135 Archive leaderboard report
Math Word Problem Solving MATH MuggleMATH-13B Parameters (Billions) 13 #87 of 135 Archive leaderboard report
Math Word Problem Solving MATH MuggleMATH 7B Accuracy 25.8 #96 of 135 Archive leaderboard report
Math Word Problem Solving MATH MuggleMATH 7B Parameters (Billions) 7 #96 of 135 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

LLaMA

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections