{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-practical-two-stage-recipe-for-mathematical","title":"A Practical Two-Stage Recipe for Mathematical LLMs: Maximizing Accuracy with SFT and Efficiency with Reinforcement Learning","arxiv_id":"2507.08267","date":"2025-07-11","proceeding":null,"authors":["Hiroshi Yoshihara","Taiki Yamaguchi","Yuichi Inoue"],"abstract":"Enhancing the mathematical reasoning of Large Language Models (LLMs) is a pivotal challenge in advancing AI capabilities. While Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) are the dominant training paradigms, a systematic methodology for combining them to maximize both accuracy and efficiency remains largely unexplored. This paper introduces a practical and effective training recipe that strategically integrates extended SFT with RL from online inference (GRPO). We posit that these methods play complementary, not competing, roles: a prolonged SFT phase first pushes the model's accuracy to its limits, after which a GRPO phase dramatically improves token efficiency while preserving this peak performance. Our experiments reveal that extending SFT for as many as 10 epochs is crucial for performance breakthroughs, and that the primary role of GRPO in this framework is to optimize solution length. The efficacy of our recipe is rigorously validated through top-tier performance on challenging benchmarks, including a high rank among over 2,200 teams in the strictly leak-free AI Mathematical Olympiad (AIMO). This work provides the community with a battle-tested blueprint for developing state-of-the-art mathematical reasoners that are both exceptionally accurate and practically efficient. To ensure full reproducibility and empower future research, we will open-source our entire framework, including all code, model checkpoints, and training configurations at https://github.com/analokmaus/kaggle-aimo2-fast-math-r1.","url_abs":"https://arxiv.org/abs/2507.08267v1","url_pdf":"https://arxiv.org/pdf/2507.08267v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-practical-two-stage-recipe-for-mathematical","repo_url":"https://github.com/analokmaus/kaggle-aimo2-fast-math-r1","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"math","task_name":"Math"},{"task_slug":"mathematical-reasoning","task_name":"Mathematical Reasoning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[{"method_slug":"sft","method_name":"SFT"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2507.08267","atlas_url":"https://app.syntology.ai/?focus=2507.08267","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2507.08267"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/analokmaus/kaggle-aimo2fast-math-r1","reach":{"status":"gone","observed_at":"2026-09-16","how":"tree_404+repo_404"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/analokmaus/kaggle-aimo2-fast-math-r1","reach":{"status":"ok"}}],"summary":{"unverified":6},"by_repo_kind":{"official":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"522a7f189e5ea3b4","entry":"create_prompt","repo":"analokmaus/kaggle-aimo2-fast-math-r1","repo_kind":"official","path":"experiments/train_fast_nemotron_14b.py","file_url":"https://github.com/analokmaus/kaggle-aimo2-fast-math-r1/blob/HEAD/experiments/train_fast_nemotron_14b.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"522a7f189e5ea3b4"}},{"code_sha256_prefix":"e14b5e417be024bf","entry":"create_prompt","repo":"analokmaus/kaggle-aimo2-fast-math-r1","repo_kind":"official","path":"experiments/train_first_stage.py","file_url":"https://github.com/analokmaus/kaggle-aimo2-fast-math-r1/blob/HEAD/experiments/train_first_stage.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e14b5e417be024bf"}},{"code_sha256_prefix":"aed7a3866e065bb8","entry":"exact_div","repo":"analokmaus/kaggle-aimo2-fast-math-r1","repo_kind":"official","path":"finetune/openr1_mod/faster_grpo_trainer.py","file_url":"https://github.com/analokmaus/kaggle-aimo2-fast-math-r1/blob/HEAD/finetune/openr1_mod/faster_grpo_trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aed7a3866e065bb8"}},{"code_sha256_prefix":"d1630e3f92e89d5d","entry":"extract_boxed_text","repo":"analokmaus/kaggle-aimo2-fast-math-r1","repo_kind":"official","path":"finetune/grpo_rewards.py","file_url":"https://github.com/analokmaus/kaggle-aimo2-fast-math-r1/blob/HEAD/finetune/grpo_rewards.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d1630e3f92e89d5d"}},{"code_sha256_prefix":"2f5695b9a0fb82de","entry":"extract_contents","repo":"analokmaus/kaggle-aimo2-fast-math-r1","repo_kind":"official","path":"finetune/grpo_rewards.py","file_url":"https://github.com/analokmaus/kaggle-aimo2-fast-math-r1/blob/HEAD/finetune/grpo_rewards.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2f5695b9a0fb82de"}},{"code_sha256_prefix":"37c8141d07e7590d","entry":"preprocess_dataset","repo":"analokmaus/kaggle-aimo2-fast-math-r1","repo_kind":"official","path":"experiments/train_token_scheduler.py","file_url":"https://github.com/analokmaus/kaggle-aimo2-fast-math-r1/blob/HEAD/experiments/train_token_scheduler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"37c8141d07e7590d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}