{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2601-04700","title":"PRISM: A Unified Framework for Post-Training LLMs Without Verifiable Rewards","arxiv_id":"2601.04700","date":"2026-01-08","proceeding":null,"authors":["Mukesh Ghimire","Aosong Feng","Liwen You","Youzhi Luo","Fang Liu","Xuan Zhu"],"abstract":"Current techniques for post-training Large Language Models (LLMs) rely either on costly human supervision or on external verifiers to boost performance on tasks such as mathematical reasoning and code generation. However, as LLMs improve their problem-solving, any further improvement will potentially require high-quality solutions to difficult problems that are not available to humans. As a result, learning from unlabeled data is becoming increasingly attractive in the research community. Existing methods extract learning signal from a model's consistency, either by majority voting or by converting the model's internal confidence into reward. Although internal consistency metric such as entropy or self-certainty require no human intervention, as we show in this work, these are unreliable signals for large-scale and long-term training. To address the unreliability, we propose PRISM, a unified training framework that uses a Process Reward Model (PRM) to guide learning alongside model's internal confidence in the absence of ground-truth labels. We show that effectively combining PRM with self-certainty can lead to both stable training and better test-time performance, and also keep the model's internal confidence in check. Code available at https://github.com/ghimiremukesh/PRISM.","url_abs":"https://arxiv.org/abs/2601.04700","url_pdf":"https://arxiv.org/pdf/2601.04700","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2601.04700","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2601.04700"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/ghimiremukesh/PRISM","reach":{"status":"ok"}}],"summary":{"ran_violates":1,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"1fa45c1d5b9a7e0a","entry":"nanmin","repo":"ghimiremukesh/PRISM","repo_kind":"found_in_text","path":"open-r1-intuitor/src/open_r1/critque_intuitor_trainer.py","file_url":"https://github.com/ghimiremukesh/PRISM/blob/HEAD/open-r1-intuitor/src/open_r1/critque_intuitor_trainer.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1fa45c1d5b9a7e0a"}},{"code_sha256_prefix":"a6cece2ffebe0f9c","entry":"nanstd","repo":"ghimiremukesh/PRISM","repo_kind":"found_in_text","path":"open-r1-intuitor/src/open_r1/critque_intuitor_trainer.py","file_url":"https://github.com/ghimiremukesh/PRISM/blob/HEAD/open-r1-intuitor/src/open_r1/critque_intuitor_trainer.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a6cece2ffebe0f9c"}},{"code_sha256_prefix":"efd67e2adf6b1ba5","entry":"split_tensor_dict","repo":"ghimiremukesh/PRISM","repo_kind":"found_in_text","path":"open-r1-intuitor/src/open_r1/critque_intuitor_trainer.py","file_url":"https://github.com/ghimiremukesh/PRISM/blob/HEAD/open-r1-intuitor/src/open_r1/critque_intuitor_trainer.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"efd67e2adf6b1ba5"}},{"code_sha256_prefix":"26d7533beea31209","entry":"extract_user_prompt","repo":"ghimiremukesh/PRISM","repo_kind":"found_in_text","path":"open-r1-intuitor/src/open_r1/decay_intuitor_prm_trainer.py","file_url":"https://github.com/ghimiremukesh/PRISM/blob/HEAD/open-r1-intuitor/src/open_r1/decay_intuitor_prm_trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"26d7533beea31209"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}