{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/internlm-math-open-math-large-language-models","title":"InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning","arxiv_id":"2402.06332","date":"2024-02-09","proceeding":null,"authors":["Huaiyuan Ying","Shuo Zhang","Linyang Li","Zhejian Zhou","Yunfan Shao","Zhaoye Fei","Yichuan Ma","Jiawei Hong","Kuikun Liu","Ziyi Wang","Yudong Wang","Zijian Wu","Shuaibin Li","Fengzhe Zhou","Hongwei Liu","Songyang Zhang","Wenwei Zhang","Hang Yan","Xipeng Qiu","Jiayu Wang","Kai Chen","Dahua Lin"],"abstract":"The math abilities of large language models can represent their abstract reasoning ability. In this paper, we introduce and open-source our math reasoning LLMs InternLM-Math which is continue pre-trained from InternLM2. We unify chain-of-thought reasoning, reward modeling, formal reasoning, data augmentation, and code interpreter in a unified seq2seq format and supervise our model to be a versatile math reasoner, verifier, prover, and augmenter. These abilities can be used to develop the next math LLMs or self-iteration. InternLM-Math obtains open-sourced state-of-the-art performance under the setting of in-context learning, supervised fine-tuning, and code-assisted reasoning in various informal and formal benchmarks including GSM8K, MATH, Hungary math exam, MathBench-ZH, and MiniF2F. Our pre-trained model achieves 30.3 on the MiniF2F test set without fine-tuning. We further explore how to use LEAN to solve math problems and study its performance under the setting of multi-task learning which shows the possibility of using LEAN as a unified platform for solving and proving in math. Our models, codes, and data are released at \\url{https://github.com/InternLM/InternLM-Math}.","url_abs":"https://arxiv.org/abs/2402.06332v2","url_pdf":"https://arxiv.org/pdf/2402.06332v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"internlm-math-open-math-large-language-models","repo_url":"https://github.com/internlm/internlm-math","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"gsm8k","task_name":"GSM8K"},{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"math","task_name":"Math"},{"task_slug":"multi-task-learning","task_name":"Multi-Task Learning"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"set","method_name":"SET"},{"method_slug":"seq2seq","method_name":"Seq2Seq"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2402.06332","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}