{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-reason-behind-good-or-bad-towards-a","title":"LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback","arxiv_id":"2406.14024","date":"2024-06-20","proceeding":null,"authors":["Bofei Gao","Zefan Cai","Runxin Xu","Peiyi Wang","Ce Zheng","Runji Lin","Keming Lu","Dayiheng Liu","Chang Zhou","Wen Xiao","Junjie Hu","Tianyu Liu","Baobao Chang"],"abstract":"In recent progress, mathematical verifiers have achieved success in mathematical reasoning tasks by validating the correctness of solutions generated by policy models. However, existing verifiers are trained with binary classification labels, which are not informative enough for the model to accurately assess the solutions. To mitigate the aforementioned insufficiency of binary labels, we introduce step-wise natural language feedback as rationale labels, that is, the correctness of each step and the detailed explanations. In this paper, we propose Math-Minos, a natural language feedback-enhanced verifier by constructing automatically generated training data and a two-stage training paradigm for effective training and efficient inference. Our experiments reveal that a small set of natural language feedback can significantly boost the performance of the verifier in both verification and reinforcement learning. We have released the code and data for further exploration.","url_abs":"https://arxiv.org/abs/2406.14024v4","url_pdf":"https://arxiv.org/pdf/2406.14024v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-reason-behind-good-or-bad-towards-a","repo_url":"https://github.com/kbsdjames/math-minos","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"binary-classification","task_name":"Binary Classification"},{"task_slug":"gsm8k","task_name":"GSM8K"},{"task_slug":"math","task_name":"Math"},{"task_slug":"mathematical-reasoning","task_name":"Mathematical Reasoning"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.14024","atlas_url":"https://app.syntology.ai/?focus=2406.14024","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.14024"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kbsdjames/math-minos","reach":{"status":"ok"}}],"summary":{"ran_violates":1,"ran_draft_wrong":4,"ran":17,"unverified":1},"by_repo_kind":{"official":{"samples":23,"ran":22,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":23,"samples":[{"code_sha256_prefix":"0fe9cf6c00ef56ea","entry":"is_number","repo":"kbsdjames/math-minos","repo_kind":"official","path":"evaluation/eval_gsm8k.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/evaluation/eval_gsm8k.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":2,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0fe9cf6c00ef56ea"}},{"code_sha256_prefix":"66b3b5450182da3a","entry":"batch_data","repo":"kbsdjames/math-minos","repo_kind":"official","path":"evaluation/eval_gsm8k.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/evaluation/eval_gsm8k.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"66b3b5450182da3a"}},{"code_sha256_prefix":"82cf8ddad3cd20d0","entry":"collate_fn","repo":"kbsdjames/math-minos","repo_kind":"official","path":"RM/process_reward_modeling_pointwise.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/RM/process_reward_modeling_pointwise.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"82cf8ddad3cd20d0"}},{"code_sha256_prefix":"f5720b2240fd9c70","entry":"collate_fn","repo":"kbsdjames/math-minos","repo_kind":"official","path":"RM/reward_modeling_pointwise.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/RM/reward_modeling_pointwise.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f5720b2240fd9c70"}},{"code_sha256_prefix":"43510407b2ec6f22","entry":"compute_metrics","repo":"kbsdjames/math-minos","repo_kind":"official","path":"RM/process_reward_modeling_pointwise.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/RM/process_reward_modeling_pointwise.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"43510407b2ec6f22"}},{"code_sha256_prefix":"307f6d829f8c616c","entry":"compute_metrics","repo":"kbsdjames/math-minos","repo_kind":"official","path":"RM/reward_modeling_pointwise.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/RM/reward_modeling_pointwise.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"307f6d829f8c616c"}},{"code_sha256_prefix":"b10fc56d74c89afd","entry":"evaluate_text_general","repo":"kbsdjames/math-minos","repo_kind":"official","path":"evaluation/meta_eval.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/evaluation/meta_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b10fc56d74c89afd"}},{"code_sha256_prefix":"0c472834a9052dd1","entry":"f1_micro","repo":"kbsdjames/math-minos","repo_kind":"official","path":"train_reward.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/train_reward.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0c472834a9052dd1"}},{"code_sha256_prefix":"f0ca9ac44e9fc7c9","entry":"find_most_frequent_answers","repo":"kbsdjames/math-minos","repo_kind":"official","path":"evaluation/eval_math.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/evaluation/eval_math.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f0ca9ac44e9fc7c9"}},{"code_sha256_prefix":"fc5fd743368dbd16","entry":"generate_training_labels","repo":"kbsdjames/math-minos","repo_kind":"official","path":"RM/process_reward_modeling_pointwise.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/RM/process_reward_modeling_pointwise.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fc5fd743368dbd16"}},{"code_sha256_prefix":"d07d04439cd1d44f","entry":"jload","repo":"kbsdjames/math-minos","repo_kind":"official","path":"train_reward.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/train_reward.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d07d04439cd1d44f"}},{"code_sha256_prefix":"8720975b15d076e8","entry":"last_boxed_only","repo":"kbsdjames/math-minos","repo_kind":"official","path":"evaluation/util.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/evaluation/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8720975b15d076e8"}},{"code_sha256_prefix":"0b14c648516c38a7","entry":"last_boxed_only_string","repo":"kbsdjames/math-minos","repo_kind":"official","path":"evaluation/util.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/evaluation/util.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0b14c648516c38a7"}},{"code_sha256_prefix":"0bbf876ced018c83","entry":"only_until_first_boxed_from_tokens","repo":"kbsdjames/math-minos","repo_kind":"official","path":"evaluation/util.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/evaluation/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0bbf876ced018c83"}},{"code_sha256_prefix":"b010240516a21945","entry":"padding","repo":"kbsdjames/math-minos","repo_kind":"official","path":"single_inference_7b_13b.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/single_inference_7b_13b.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b010240516a21945"}},{"code_sha256_prefix":"2128e09f82896741","entry":"prepare_model_input","repo":"kbsdjames/math-minos","repo_kind":"official","path":"RM/reward_model_inference_MATH.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/RM/reward_model_inference_MATH.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2128e09f82896741"}},{"code_sha256_prefix":"205133e22aeebca1","entry":"prepare_model_input","repo":"kbsdjames/math-minos","repo_kind":"official","path":"RM/reward_model_inference.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/RM/reward_model_inference.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"205133e22aeebca1"}},{"code_sha256_prefix":"51c01dd0d033f660","entry":"preprocess","repo":"kbsdjames/math-minos","repo_kind":"official","path":"train_reward.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/train_reward.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"51c01dd0d033f660"}},{"code_sha256_prefix":"5e7416d3c1bc76ee","entry":"preprocess","repo":"kbsdjames/math-minos","repo_kind":"official","path":"single_inference_7b_13b.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/single_inference_7b_13b.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5e7416d3c1bc76ee"}},{"code_sha256_prefix":"b776183e27cffbb8","entry":"process_model_output","repo":"kbsdjames/math-minos","repo_kind":"official","path":"evaluation/meta_eval.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/evaluation/meta_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b776183e27cffbb8"}},{"code_sha256_prefix":"f3bbe264b05aadd3","entry":"remove_boxed","repo":"kbsdjames/math-minos","repo_kind":"official","path":"evaluation/eval_math.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/evaluation/eval_math.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f3bbe264b05aadd3"}},{"code_sha256_prefix":"ef8c2d16d11565e0","entry":"tokenize_examples","repo":"kbsdjames/math-minos","repo_kind":"official","path":"RM/reward_modeling_pointwise.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/RM/reward_modeling_pointwise.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ef8c2d16d11565e0"}},{"code_sha256_prefix":"ca3987a32591915a","entry":"sequence_gather","repo":"kbsdjames/math-minos","repo_kind":"official","path":"single_inference_7b_13b.py","file_url":"https://github.com/kbsdjames/math-minos/blob/HEAD/single_inference_7b_13b.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ca3987a32591915a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}