{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/autocv-empowering-reasoning-with-automated","title":"AutoPSV: Automated Process-Supervised Verifier","arxiv_id":"2405.16802","date":"2024-05-27","proceeding":null,"authors":["Jianqiao Lu","Zhiyang Dou","Hongru Wang","Zeyu Cao","Jianbo Dai","Yingjia Wan","Zhijiang Guo"],"abstract":"In this work, we propose a novel method named \\textbf{Auto}mated \\textbf{P}rocess-\\textbf{S}upervised \\textbf{V}erifier (\\textbf{\\textsc{AutoPSV}}) to enhance the reasoning capabilities of large language models (LLMs) by automatically annotating the reasoning steps. \\textsc{AutoPSV} begins by training a verification model on the correctness of final answers, enabling it to generate automatic process annotations. This verification model assigns a confidence score to each reasoning step, indicating the probability of arriving at the correct final answer from that point onward. We detect relative changes in the verification's confidence scores across reasoning steps to automatically annotate the reasoning process, enabling error detection even in scenarios where ground truth answers are unavailable. This alleviates the need for numerous manual annotations or the high computational costs associated with model-induced annotation approaches. We experimentally validate that the step-level confidence changes learned by the verification model trained on the final answer correctness can effectively identify errors in the reasoning steps. We demonstrate that the verification model, when trained on process annotations generated by \\textsc{AutoPSV}, exhibits improved performance in selecting correct answers from multiple LLM-generated outputs. Notably, we achieve substantial improvements across five datasets in mathematics and commonsense reasoning. The source code of \\textsc{AutoPSV} is available at \\url{https://github.com/rookie-joe/AutoPSV}.","url_abs":"https://arxiv.org/abs/2405.16802v4","url_pdf":"https://arxiv.org/pdf/2405.16802v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"autocv-empowering-reasoning-with-automated","repo_url":"https://github.com/rookie-joe/autocv","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"autocv-empowering-reasoning-with-automated","repo_url":"https://github.com/rookie-joe/autopsv","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.16802","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.16802"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/rookie-joe/AutoPSV","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rookie-joe/autocv","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rookie-joe/autopsv","reach":null}],"summary":{"ran":5,"ran_draft_wrong":2,"ran_fixture":1,"unverified":3},"by_repo_kind":{"official":{"samples":11,"ran":8,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"31531bb131326426","entry":"Verifier","repo":"rookie-joe/autopsv","repo_kind":"official","path":"utils/process_verifier_models.py","file_url":"https://github.com/rookie-joe/autopsv/blob/HEAD/utils/process_verifier_models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"31531bb131326426"}},{"code_sha256_prefix":"49640af61a6d3492","entry":"create_train_dataloader","repo":"rookie-joe/autocv","repo_kind":"official","path":"utils/verfier_datasets.py","file_url":"https://github.com/rookie-joe/autocv/blob/HEAD/utils/verfier_datasets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"49640af61a6d3492"}},{"code_sha256_prefix":"69664c2b879ea002","entry":"get_dataset_examples","repo":"rookie-joe/autocv","repo_kind":"official","path":"utils/verfier_datasets.py","file_url":"https://github.com/rookie-joe/autocv/blob/HEAD/utils/verfier_datasets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"69664c2b879ea002"}},{"code_sha256_prefix":"17243183193565ff","entry":"load_tokenizer","repo":"rookie-joe/autocv","repo_kind":"official","path":"data_annotation.py","file_url":"https://github.com/rookie-joe/autocv/blob/HEAD/data_annotation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"17243183193565ff"}},{"code_sha256_prefix":"dfef91c8dfd2a1ee","entry":"mse_loss_with_mask","repo":"rookie-joe/autopsv","repo_kind":"official","path":"utils/process_verifier_models.py","file_url":"https://github.com/rookie-joe/autopsv/blob/HEAD/utils/process_verifier_models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dfef91c8dfd2a1ee"}},{"code_sha256_prefix":"c409d17fa33773fa","entry":"read_jsonl","repo":"rookie-joe/autocv","repo_kind":"official","path":"utils/verfier_datasets.py","file_url":"https://github.com/rookie-joe/autocv/blob/HEAD/utils/verfier_datasets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c409d17fa33773fa"}},{"code_sha256_prefix":"8fd49b81e935073b","entry":"split_string_list","repo":"rookie-joe/AutoPSV","repo_kind":"official","path":"data_annotation.py","file_url":"https://github.com/rookie-joe/AutoPSV/blob/HEAD/data_annotation.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8fd49b81e935073b"}},{"code_sha256_prefix":"3f4f58ae7ef13d89","entry":"split_token_list","repo":"rookie-joe/AutoPSV","repo_kind":"official","path":"data_annotation.py","file_url":"https://github.com/rookie-joe/AutoPSV/blob/HEAD/data_annotation.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3f4f58ae7ef13d89"}},{"code_sha256_prefix":"c8b43628f651b52e","entry":"VerifierModelOutput","repo":"rookie-joe/autopsv","repo_kind":"official","path":"utils/process_verifier_models.py","file_url":"https://github.com/rookie-joe/autopsv/blob/HEAD/utils/process_verifier_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c8b43628f651b52e"}},{"code_sha256_prefix":"100d6596edff1631","entry":"build_verifier_from_osv","repo":"rookie-joe/autocv","repo_kind":"official","path":"utils/process_verifier_models.py","file_url":"https://github.com/rookie-joe/autocv/blob/HEAD/utils/process_verifier_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"100d6596edff1631"}},{"code_sha256_prefix":"e460ed999d4fa5b0","entry":"build_verifier_from_scratch","repo":"rookie-joe/autocv","repo_kind":"official","path":"utils/process_verifier_models.py","file_url":"https://github.com/rookie-joe/autocv/blob/HEAD/utils/process_verifier_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e460ed999d4fa5b0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}