{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/breaking-language-barriers-in-multilingual","title":"Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations","arxiv_id":"2310.20246","date":"2023-10-31","proceeding":null,"authors":["Nuo Chen","Zinan Zheng","Ning Wu","Ming Gong","Dongmei Zhang","Jia Li"],"abstract":"Existing research predominantly focuses on developing powerful language learning models (LLMs) for mathematical reasoning within monolingual languages, with few explorations in preserving efficacy in a multilingual context. To bridge this gap, this paper pioneers exploring and training powerful Multilingual Math Reasoning (xMR) LLMs. Firstly, by utilizing translation, we construct the first multilingual math reasoning instruction dataset, MGSM8KInstruct, encompassing ten distinct languages, thus addressing the issue of training data scarcity in xMR tasks. Based on the collected dataset, we propose different training strategies to build powerful xMR LLMs, named MathOctopus, notably outperform conventional open-source LLMs and exhibit superiority over ChatGPT in few-shot scenarios. Notably, MathOctopus-13B reaches 47.6% accuracy which exceeds ChatGPT 46.3% on MGSM testset. Beyond remarkable results, we unearth several pivotal observations and insights from extensive experiments: (1) When extending the rejection sampling strategy to the multilingual context, it proves effective for model performances, albeit limited. (2) Employing parallel corpora for math Supervised Fine-Tuning (SFT) across multiple languages not only significantly enhances model performance multilingually but also elevates their monolingual performance. This indicates that crafting multilingual corpora can be regarded as a vital strategy for enhancing model performance in a specific language, especially in mathematical reasoning tasks. For instance, MathOctopus-7B improves its counterparts that trained on English from 42.2% to 50.8% on GSM8K testset. Codes are available at https://github.com/microsoft/MathOctopus.","url_abs":"https://arxiv.org/abs/2310.20246v5","url_pdf":"https://arxiv.org/pdf/2310.20246v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"breaking-language-barriers-in-multilingual","repo_url":"https://github.com/microsoft/MathOctopus","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"breaking-language-barriers-in-multilingual","repo_url":"https://github.com/laihuiyuan/mcot","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"gsm8k","task_name":"GSM8K"},{"task_slug":"math","task_name":"Math"},{"task_slug":"mathematical-reasoning","task_name":"Mathematical Reasoning"}],"methods":[],"datasets_introduced":[{"slug":"mgsm8kinstruct","name":"MGSM8KInstruct","full_name":"MGSM8KInstruct"},{"slug":"msvamp","name":"MSVAMP","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.20246","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.20246"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/microsoft/MathOctopus","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/laihuiyuan/mcot","reach":{"status":"ok"}}],"summary":{"ran":7,"unverified":4},"by_repo_kind":{"official":{"samples":11,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"451c619b9609c3c0","entry":"generate","repo":"microsoft/MathOctopus","repo_kind":"official","path":"step1_supervised_finetuning/prompt_eval.py","file_url":"https://github.com/microsoft/MathOctopus/blob/HEAD/step1_supervised_finetuning/prompt_eval.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"451c619b9609c3c0"}},{"code_sha256_prefix":"385922750eb70702","entry":"generate_constrastive_search","repo":"microsoft/MathOctopus","repo_kind":"official","path":"step1_supervised_finetuning/prompt_eval.py","file_url":"https://github.com/microsoft/MathOctopus/blob/HEAD/step1_supervised_finetuning/prompt_eval.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"385922750eb70702"}},{"code_sha256_prefix":"2e064bd50d861c4d","entry":"get_batch_llama","repo":"microsoft/MathOctopus","repo_kind":"official","path":"step1_supervised_finetuning/generate_and_eval.py","file_url":"https://github.com/microsoft/MathOctopus/blob/HEAD/step1_supervised_finetuning/generate_and_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2e064bd50d861c4d"}},{"code_sha256_prefix":"7f4bdf1898ad0544","entry":"get_eval_ds_config","repo":"microsoft/MathOctopus","repo_kind":"official","path":"utils/ds_utils.py","file_url":"https://github.com/microsoft/MathOctopus/blob/HEAD/utils/ds_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f4bdf1898ad0544"}},{"code_sha256_prefix":"311aba0a9b717df8","entry":"get_train_ds_config","repo":"microsoft/MathOctopus","repo_kind":"official","path":"utils/ds_utils.py","file_url":"https://github.com/microsoft/MathOctopus/blob/HEAD/utils/ds_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"311aba0a9b717df8"}},{"code_sha256_prefix":"52f51e3b99457a09","entry":"gsm8k_batch_gen","repo":"microsoft/MathOctopus","repo_kind":"official","path":"step1_supervised_finetuning/generate_and_eval.py","file_url":"https://github.com/microsoft/MathOctopus/blob/HEAD/step1_supervised_finetuning/generate_and_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"52f51e3b99457a09"}},{"code_sha256_prefix":"e617041853afd002","entry":"only_optimize_lora_parameters","repo":"microsoft/MathOctopus","repo_kind":"official","path":"utils/module/lora.py","file_url":"https://github.com/microsoft/MathOctopus/blob/HEAD/utils/module/lora.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e617041853afd002"}},{"code_sha256_prefix":"016d55f875a4163a","entry":"get_model","repo":"microsoft/MathOctopus","repo_kind":"official","path":"step1_supervised_finetuning/generate_and_eval.py","file_url":"https://github.com/microsoft/MathOctopus/blob/HEAD/step1_supervised_finetuning/generate_and_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"016d55f875a4163a"}},{"code_sha256_prefix":"c791912844fe37e5","entry":"get_tokenizer","repo":"microsoft/MathOctopus","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/microsoft/MathOctopus/blob/HEAD/utils/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c791912844fe37e5"}},{"code_sha256_prefix":"4573c4150d38e483","entry":"load_hf_tokenizer","repo":"microsoft/MathOctopus","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/microsoft/MathOctopus/blob/HEAD/utils/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4573c4150d38e483"}},{"code_sha256_prefix":"387b9e6c4f21dd62","entry":"to_device","repo":"microsoft/MathOctopus","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/microsoft/MathOctopus/blob/HEAD/utils/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"387b9e6c4f21dd62"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}