{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mwp-bert-a-strong-baseline-for-math-word","title":"MWP-BERT: Numeracy-Augmented Pre-training for Math Word Problem Solving","arxiv_id":"2107.13435","date":"2021-07-28","proceeding":"Findings (NAACL) 2022 7","authors":["Zhenwen Liang","Jipeng Zhang","Lei Wang","Wei Qin","Yunshi Lan","Jie Shao","Xiangliang Zhang"],"abstract":"Math word problem (MWP) solving faces a dilemma in number representation learning. In order to avoid the number representation issue and reduce the search space of feasible solutions, existing works striving for MWP solving usually replace real numbers with symbolic placeholders to focus on logic reasoning. However, different from common symbolic reasoning tasks like program synthesis and knowledge graph reasoning, MWP solving has extra requirements in numerical reasoning. In other words, instead of the number value itself, it is the reusable numerical property that matters more in numerical reasoning. Therefore, we argue that injecting numerical properties into symbolic placeholders with contextualized representation learning schema can provide a way out of the dilemma in the number representation issue here. In this work, we introduce this idea to the popular pre-training language model (PLM) techniques and build MWP-BERT, an effective contextual number representation PLM. We demonstrate the effectiveness of our MWP-BERT on MWP solving and several MWP-specific understanding tasks on both English and Chinese benchmarks.","url_abs":"https://arxiv.org/abs/2107.13435v2","url_pdf":"https://arxiv.org/pdf/2107.13435v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mwp-bert-a-strong-baseline-for-math-word","repo_url":"https://github.com/lzhenwen/mwp-bert","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"common-sense-reasoning","task_name":"Common Sense Reasoning"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"math","task_name":"Math"},{"task_slug":"math-word-problem-solving","task_name":"Math Word Problem Solving"},{"task_slug":"program-synthesis","task_name":"Program Synthesis"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/math-word-problem-solving-on-math23k","task":"Math Word Problem Solving","dataset":"Math23K","model":"MWP-BERT","rank_in_archive_order":7,"of":19,"metrics":{"Accuracy (5-fold)":"82.4","Accuracy (training-test)":"84.7"},"uses_additional_data":false},{"leaderboard":"/sota/math-word-problem-solving-on-mathqa","task":"Math Word Problem Solving","dataset":"MathQA","model":"MWP-BERT","rank_in_archive_order":5,"of":5,"metrics":{"Answer Accuracy":"76.6"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2107.13435","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2107.13435"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lzhenwen/mwp-bert","reach":{"status":"ok"}}],"summary":{"ran_honours":1,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"a8fc981dd5a6a740","entry":"change_num","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"a8fc981dd5a6a740"}},{"code_sha256_prefix":"c31a919f85cca48c","entry":"read_json","repo":"lzhenwen/mwp-bert","repo_kind":"official","path":"Fine-tuning/Math23k/math23k.py","file_url":"https://github.com/lzhenwen/mwp-bert/blob/HEAD/Fine-tuning/Math23k/math23k.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c31a919f85cca48c"}},{"code_sha256_prefix":"c7b9a386e557baa2","entry":"initial_model","repo":"lzhenwen/mwp-bert","repo_kind":"official","path":"Fine-tuning/MathQA/run_ft.py","file_url":"https://github.com/lzhenwen/mwp-bert/blob/HEAD/Fine-tuning/MathQA/run_ft.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c7b9a386e557baa2"}},{"code_sha256_prefix":"d8e7137b458da412","entry":"make_pair","repo":"lzhenwen/mwp-bert","repo_kind":"official","path":"Fine-tuning/MathQA/run_ft.py","file_url":"https://github.com/lzhenwen/mwp-bert/blob/HEAD/Fine-tuning/MathQA/run_ft.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d8e7137b458da412"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}