{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mathprompter-mathematical-reasoning-using","title":"MathPrompter: Mathematical Reasoning using Large Language Models","arxiv_id":"2303.05398","date":"2023-03-04","proceeding":null,"authors":["Shima Imani","Liang Du","Harsh Shrivastava"],"abstract":"Large Language Models (LLMs) have limited performance when solving arithmetic reasoning tasks and often provide incorrect answers. Unlike natural language understanding, math problems typically have a single correct answer, making the task of generating accurate solutions more challenging for LLMs. To the best of our knowledge, we are not aware of any LLMs that indicate their level of confidence in their responses which fuels a trust deficit in these models impeding their adoption. To address this deficiency, we propose `MathPrompter', a technique that improves performance of LLMs on arithmetic problems along with increased reliance in the predictions. MathPrompter uses the Zero-shot chain-of-thought prompting technique to generate multiple Algebraic expressions or Python functions to solve the same math problem in different ways and thereby raise the confidence level in the output results. This is in contrast to other prompt based CoT methods, where there is no check on the validity of the intermediate steps followed. Our technique improves over state-of-the-art on the MultiArith dataset ($78.7\\%\\rightarrow92.5\\%$) evaluated using 175B parameter GPT-based LLM.","url_abs":"https://arxiv.org/abs/2303.05398v1","url_pdf":"https://arxiv.org/pdf/2303.05398v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mathprompter-mathematical-reasoning-using","repo_url":"https://github.com/RamonKaspar/MathPrompter","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"arithmetic-reasoning","task_name":"Arithmetic Reasoning"},{"task_slug":"math","task_name":"Math"},{"task_slug":"mathematical-reasoning","task_name":"Mathematical Reasoning"},{"task_slug":"natural-language-understanding","task_name":"Natural Language Understanding"}],"methods":[{"method_slug":"aware","method_name":"AWARE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.05398","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.05398"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/RamonKaspar/MathPrompter","reach":null}],"summary":{"ran":2,"unverified":3},"by_repo_kind":{"listed":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"da81018d427bc031","entry":"OpenAIService","repo":"RamonKaspar/MathPrompter","repo_kind":"listed","path":"math_prompts_generation.py","file_url":"https://github.com/RamonKaspar/MathPrompter/blob/HEAD/math_prompts_generation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"da81018d427bc031"}},{"code_sha256_prefix":"bf1b6f89bf903fbc","entry":"get_llm_service","repo":"RamonKaspar/MathPrompter","repo_kind":"listed","path":"math_prompts_generation.py","file_url":"https://github.com/RamonKaspar/MathPrompter/blob/HEAD/math_prompts_generation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bf1b6f89bf903fbc"}},{"code_sha256_prefix":"905603ae58189fc7","entry":"AzureOpenAIService","repo":"RamonKaspar/MathPrompter","repo_kind":"listed","path":"math_prompts_generation.py","file_url":"https://github.com/RamonKaspar/MathPrompter/blob/HEAD/math_prompts_generation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"905603ae58189fc7"}},{"code_sha256_prefix":"06c040ff02496ef2","entry":"LLMInterface","repo":"RamonKaspar/MathPrompter","repo_kind":"listed","path":"math_prompts_generation.py","file_url":"https://github.com/RamonKaspar/MathPrompter/blob/HEAD/math_prompts_generation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"06c040ff02496ef2"}},{"code_sha256_prefix":"907831f47445bd3f","entry":"algebric_expression_generation","repo":"RamonKaspar/MathPrompter","repo_kind":"listed","path":"math_prompts_generation.py","file_url":"https://github.com/RamonKaspar/MathPrompter/blob/HEAD/math_prompts_generation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"907831f47445bd3f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}