{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/absolute-zero-reinforced-self-play-reasoning","title":"Absolute Zero: Reinforced Self-play Reasoning with Zero Data","arxiv_id":"2505.03335","date":"2025-05-06","proceeding":null,"authors":["Andrew Zhao","Yiran Wu","Tong Wu","Quentin Xu","Yang Yue","Matthieu Lin","Shenzhi Wang","Qingyun Wu","Zilong Zheng","Gao Huang"],"abstract":"Reinforcement learning with verifiable rewards (RLVR) has shown promise in enhancing the reasoning capabilities of large language models by learning directly from outcome-based rewards. Recent RLVR works that operate under the zero setting avoid supervision in labeling the reasoning process, but still depend on manually curated collections of questions and answers for training. The scarcity of high-quality, human-produced examples raises concerns about the long-term scalability of relying on human supervision, a challenge already evident in the domain of language model pretraining. Furthermore, in a hypothetical future where AI surpasses human intelligence, tasks provided by humans may offer limited learning potential for a superintelligent system. To address these concerns, we propose a new RLVR paradigm called Absolute Zero, in which a single model learns to propose tasks that maximize its own learning progress and improves reasoning by solving them, without relying on any external data. Under this paradigm, we introduce the Absolute Zero Reasoner (AZR), a system that self-evolves its training curriculum and reasoning ability by using a code executor to both validate proposed code reasoning tasks and verify answers, serving as an unified source of verifiable reward to guide open-ended yet grounded learning. Despite being trained entirely without external data, AZR achieves overall SOTA performance on coding and mathematical reasoning tasks, outperforming existing zero-setting models that rely on tens of thousands of in-domain human-curated examples. Furthermore, we demonstrate that AZR can be effectively applied across different model scales and is compatible with various model classes.","url_abs":"https://arxiv.org/abs/2505.03335v2","url_pdf":"https://arxiv.org/pdf/2505.03335v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"absolute-zero-reinforced-self-play-reasoning","repo_url":"https://github.com/LeapLabTHU/Absolute-Zero-Reasoner","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"absolute-zero-reinforced-self-play-reasoning","repo_url":"https://github.com/volcengine/verl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"absolute-zero-reinforced-self-play-reasoning","repo_url":"https://github.com/qingyangzhang/empo","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"mathematical-reasoning","task_name":"Mathematical Reasoning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2505.03335","atlas_url":"https://app.syntology.ai/?focus=2505.03335","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.03335"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/volcengine/verl","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qingyangzhang/empo","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/LeapLabTHU/Absolute-Zero-Reasoner","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1,"ran_draft_wrong":1,"ran":2,"ran_violates":1,"unverified":7},"by_repo_kind":{"official":{"samples":12,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"368d3db6347269c3","entry":"count_unknown_letters_in_expr","repo":"LeapLabTHU/Absolute-Zero-Reasoner","repo_kind":"official","path":"absolute_zero_reasoner/rewards/math_utils.py","file_url":"https://github.com/LeapLabTHU/Absolute-Zero-Reasoner/blob/HEAD/absolute_zero_reasoner/rewards/math_utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"368d3db6347269c3"}},{"code_sha256_prefix":"dd14950a6b76ffbc","entry":"extract_code","repo":"LeapLabTHU/Absolute-Zero-Reasoner","repo_kind":"official","path":"absolute_zero_reasoner/rewards/code_reward.py","file_url":"https://github.com/LeapLabTHU/Absolute-Zero-Reasoner/blob/HEAD/absolute_zero_reasoner/rewards/code_reward.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dd14950a6b76ffbc"}},{"code_sha256_prefix":"72c1465a54867eee","entry":"extract_code","repo":"LeapLabTHU/Absolute-Zero-Reasoner","repo_kind":"official","path":"absolute_zero_reasoner/rewards/custom_evaluate.py","file_url":"https://github.com/LeapLabTHU/Absolute-Zero-Reasoner/blob/HEAD/absolute_zero_reasoner/rewards/custom_evaluate.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"72c1465a54867eee"}},{"code_sha256_prefix":"c077759d0f068d30","entry":"process_livecodebench_execution","repo":"LeapLabTHU/Absolute-Zero-Reasoner","repo_kind":"official","path":"absolute_zero_reasoner/data_construction/process_code_reasoning_data.py","file_url":"https://github.com/LeapLabTHU/Absolute-Zero-Reasoner/blob/HEAD/absolute_zero_reasoner/data_construction/process_code_reasoning_data.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c077759d0f068d30"}},{"code_sha256_prefix":"95f02454eddf082a","entry":"should_allow_eval","repo":"LeapLabTHU/Absolute-Zero-Reasoner","repo_kind":"official","path":"absolute_zero_reasoner/rewards/math_utils.py","file_url":"https://github.com/LeapLabTHU/Absolute-Zero-Reasoner/blob/HEAD/absolute_zero_reasoner/rewards/math_utils.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"95f02454eddf082a"}},{"code_sha256_prefix":"7dadfcd516d43505","entry":"add_imports","repo":"LeapLabTHU/Absolute-Zero-Reasoner","repo_kind":"official","path":"absolute_zero_reasoner/data_construction/process_code_reasoning_data.py","file_url":"https://github.com/LeapLabTHU/Absolute-Zero-Reasoner/blob/HEAD/absolute_zero_reasoner/data_construction/process_code_reasoning_data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7dadfcd516d43505"}},{"code_sha256_prefix":"0742e5f056e04404","entry":"choice_answer_clean","repo":"LeapLabTHU/Absolute-Zero-Reasoner","repo_kind":"official","path":"absolute_zero_reasoner/rewards/custom_evaluate.py","file_url":"https://github.com/LeapLabTHU/Absolute-Zero-Reasoner/blob/HEAD/absolute_zero_reasoner/rewards/custom_evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0742e5f056e04404"}},{"code_sha256_prefix":"a08febbb4b8dc4c6","entry":"get_code_problem_generator_prompt","repo":"LeapLabTHU/Absolute-Zero-Reasoner","repo_kind":"official","path":"absolute_zero_reasoner/data_construction/prompts.py","file_url":"https://github.com/LeapLabTHU/Absolute-Zero-Reasoner/blob/HEAD/absolute_zero_reasoner/data_construction/prompts.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a08febbb4b8dc4c6"}},{"code_sha256_prefix":"1b213fd26ce83e1b","entry":"get_code_problem_predictor_prompt","repo":"LeapLabTHU/Absolute-Zero-Reasoner","repo_kind":"official","path":"absolute_zero_reasoner/data_construction/prompts.py","file_url":"https://github.com/LeapLabTHU/Absolute-Zero-Reasoner/blob/HEAD/absolute_zero_reasoner/data_construction/prompts.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1b213fd26ce83e1b"}},{"code_sha256_prefix":"ec578739f88adc22","entry":"mathd_normalize_answer","repo":"LeapLabTHU/Absolute-Zero-Reasoner","repo_kind":"official","path":"absolute_zero_reasoner/rewards/math_utils.py","file_url":"https://github.com/LeapLabTHU/Absolute-Zero-Reasoner/blob/HEAD/absolute_zero_reasoner/rewards/math_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec578739f88adc22"}},{"code_sha256_prefix":"48586c02e33616fc","entry":"parse_to_ast","repo":"LeapLabTHU/Absolute-Zero-Reasoner","repo_kind":"official","path":"absolute_zero_reasoner/rewards/code_reward.py","file_url":"https://github.com/LeapLabTHU/Absolute-Zero-Reasoner/blob/HEAD/absolute_zero_reasoner/rewards/code_reward.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"48586c02e33616fc"}},{"code_sha256_prefix":"d1604f4eadcbe65a","entry":"run_huggingface_download","repo":"LeapLabTHU/Absolute-Zero-Reasoner","repo_kind":"official","path":"absolute_zero_reasoner/utils/remove_think_qwen3_tokenizer.py","file_url":"https://github.com/LeapLabTHU/Absolute-Zero-Reasoner/blob/HEAD/absolute_zero_reasoner/utils/remove_think_qwen3_tokenizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d1604f4eadcbe65a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}