{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rethinking-optimal-verification-granularity","title":"Rethinking Optimal Verification Granularity for Compute-Efficient Test-Time Scaling","arxiv_id":"2505.11730","date":"2025-05-16","proceeding":null,"authors":["Hao Mark Chen","Guanxi Lu","Yasuyuki Okoshi","Zhiwen Mo","Masato Motomura","Hongxiang Fan"],"abstract":"Test-time scaling (TTS) has proven effective in enhancing the reasoning capabilities of large language models (LLMs). Verification plays a key role in TTS, simultaneously influencing (1) reasoning performance and (2) compute efficiency, due to the quality and computational cost of verification. In this work, we challenge the conventional paradigms of verification, and make the first attempt toward systematically investigating the impact of verification granularity-that is, how frequently the verifier is invoked during generation, beyond verifying only the final output or individual generation steps. To this end, we introduce Variable Granularity Search (VG-Search), a unified algorithm that generalizes beam search and Best-of-N sampling via a tunable granularity parameter g. Extensive experiments with VG-Search under varying compute budgets, generator-verifier configurations, and task attributes reveal that dynamically selecting g can improve the compute efficiency and scaling behavior. Building on these findings, we propose adaptive VG-Search strategies that achieve accuracy gains of up to 3.1\\% over Beam Search and 3.6\\% over Best-of-N, while reducing FLOPs by over 52\\%. We will open-source the code to support future research.","url_abs":"https://arxiv.org/abs/2505.11730v1","url_pdf":"https://arxiv.org/pdf/2505.11730v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2505.11730","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.11730"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/hmarkc/VG-Search","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":8},"by_repo_kind":{"found_in_text":{"samples":8,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"e2fa8778aaa14477","entry":"aggregate_scores","repo":"hmarkc/VG-Search","repo_kind":"found_in_text","path":"src/sal/utils/score.py","file_url":"https://github.com/hmarkc/VG-Search/blob/HEAD/src/sal/utils/score.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"e2fa8778aaa14477"}},{"code_sha256_prefix":"fcb58cf3620ba3fa","entry":"build_conv","repo":"hmarkc/VG-Search","repo_kind":"found_in_text","path":"src/sal/search/utils.py","file_url":"https://github.com/hmarkc/VG-Search/blob/HEAD/src/sal/search/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"fcb58cf3620ba3fa"}},{"code_sha256_prefix":"3c4ade317b7b432b","entry":"extract_completion_answers","repo":"hmarkc/VG-Search","repo_kind":"found_in_text","path":"src/sal/utils/math.py","file_url":"https://github.com/hmarkc/VG-Search/blob/HEAD/src/sal/utils/math.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"3c4ade317b7b432b"}},{"code_sha256_prefix":"ed2f4689128f9885","entry":"extract_multi_choice_answer","repo":"hmarkc/VG-Search","repo_kind":"found_in_text","path":"src/sal/utils/qwen_math_parser.py","file_url":"https://github.com/hmarkc/VG-Search/blob/HEAD/src/sal/utils/qwen_math_parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"ed2f4689128f9885"}},{"code_sha256_prefix":"07e55a87bf057096","entry":"get_dataset_revisions","repo":"hmarkc/VG-Search","repo_kind":"found_in_text","path":"src/sal/utils/hub.py","file_url":"https://github.com/hmarkc/VG-Search/blob/HEAD/src/sal/utils/hub.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"07e55a87bf057096"}},{"code_sha256_prefix":"0a2729199ea72cd2","entry":"last","repo":"hmarkc/VG-Search","repo_kind":"found_in_text","path":"src/sal/search/utils.py","file_url":"https://github.com/hmarkc/VG-Search/blob/HEAD/src/sal/search/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"0a2729199ea72cd2"}},{"code_sha256_prefix":"e1583186b85bad63","entry":"list_mean","repo":"hmarkc/VG-Search","repo_kind":"found_in_text","path":"src/sal/search/utils.py","file_url":"https://github.com/hmarkc/VG-Search/blob/HEAD/src/sal/search/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"e1583186b85bad63"}},{"code_sha256_prefix":"940f44544351a853","entry":"subsample_completions","repo":"hmarkc/VG-Search","repo_kind":"found_in_text","path":"src/sal/utils/math.py","file_url":"https://github.com/hmarkc/VG-Search/blob/HEAD/src/sal/utils/math.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"940f44544351a853"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}