{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2602-11570","title":"PRIME: A Process-Outcome Alignment Benchmark for Verifiable Reasoning in Mathematics and Engineering","arxiv_id":"2602.11570","date":"2026-02-12","proceeding":null,"authors":["Xiangfeng Wang","Hangyu Guo","Yanlin Lai","Mitt Huang","Liang Zhao","Chengyuan Yao","Yinmin Zhang","Qi Han","Xiaoxiao Ren","Chun Yuan","Tong Xu","Zheng Ge","Xiangyu Zhang","Daxin Jiang"],"abstract":"While model-based verifiers are essential for scaling Reinforcement Learning with Verifiable Rewards (RLVR), current outcome-centric verification paradigms primarily focus on the consistency between the final result and the ground truth, often neglecting potential errors in the derivation process. This leads to assigning positive rewards to correct answers produced from incorrect derivations. To bridge this gap, we introduce PRIME, a benchmark for evaluating verifiers on Process-Outcome Alignment verification in Mathematics and Engineering. Curated from a comprehensive collection of college-level STEM problems, PRIME comprises 2,530 high-difficulty samples through a consistency-based filtering pipeline. Through extensive evaluation, we find that current verifiers frequently fail to detect derivation flaws. Furthermore, we propose a process-aware RLVR training paradigm utilizing verifiers selected via PRIME. This approach substantially outperforms the outcome-only verification baseline, achieving absolute performance gains of 8.29%, 9.12%, and 7.31% on AIME24, AIME25, and Beyond-AIME, respectively, for the Qwen3-14B-Base model. Finally, we demonstrate a strong linear correlation ($R^2 > 0.92$) between verifier accuracy on PRIME and RLVR training effectiveness, validating PRIME as a reliable predictor for verifier selection.","url_abs":"https://arxiv.org/abs/2602.11570","url_pdf":"https://arxiv.org/pdf/2602.11570","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2602.11570","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2602.11570"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/wonderful9462/PRIME","reach":null}],"summary":{"unverified":4},"by_repo_kind":{"found_in_text":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"32f5344f9c7eac24","entry":"calculate_scores","repo":"wonderful9462/PRIME","repo_kind":"found_in_text","path":"evaluation/calculate_accuracy.py","file_url":"https://github.com/wonderful9462/PRIME/blob/HEAD/evaluation/calculate_accuracy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"32f5344f9c7eac24"}},{"code_sha256_prefix":"e46a2c41657f58df","entry":"calculate_token_number","repo":"wonderful9462/PRIME","repo_kind":"found_in_text","path":"evaluation/calculate_accuracy.py","file_url":"https://github.com/wonderful9462/PRIME/blob/HEAD/evaluation/calculate_accuracy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e46a2c41657f58df"}},{"code_sha256_prefix":"59a153f4788626a3","entry":"load_jsonl_file","repo":"wonderful9462/PRIME","repo_kind":"found_in_text","path":"evaluation/calculate_accuracy.py","file_url":"https://github.com/wonderful9462/PRIME/blob/HEAD/evaluation/calculate_accuracy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"59a153f4788626a3"}},{"code_sha256_prefix":"e1ae9a61eea7cd2c","entry":"process_judgment","repo":"wonderful9462/PRIME","repo_kind":"found_in_text","path":"evaluation/evaluate_compassverifier.py","file_url":"https://github.com/wonderful9462/PRIME/blob/HEAD/evaluation/evaluate_compassverifier.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e1ae9a61eea7cd2c"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}