{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adactrl-towards-adaptive-and-controllable","title":"AdaCtrl: Towards Adaptive and Controllable Reasoning via Difficulty-Aware Budgeting","arxiv_id":"2505.18822","date":"2025-05-24","proceeding":null,"authors":["Shijue Huang","Hongru Wang","Wanjun Zhong","Zhaochen Su","Jiazhan Feng","Bowen Cao","Yi R. Fung"],"abstract":"Modern large reasoning models demonstrate impressive problem-solving capabilities by employing sophisticated reasoning strategies. However, they often struggle to balance efficiency and effectiveness, frequently generating unnecessarily lengthy reasoning chains for simple problems. In this work, we propose AdaCtrl, a novel framework to support both difficulty-aware adaptive reasoning budget allocation and explicit user control over reasoning depth. AdaCtrl dynamically adjusts its reasoning length based on self-assessed problem difficulty, while also allowing users to manually control the budget to prioritize either efficiency or effectiveness. This is achieved through a two-stage training pipeline: an initial cold-start fine-tuning phase to instill the ability to self-aware difficulty and adjust reasoning budget, followed by a difficulty-aware reinforcement learning (RL) stage that refines the model's adaptive reasoning strategies and calibrates its difficulty assessments based on its evolving capabilities during online training. To enable intuitive user interaction, we design explicit length-triggered tags that function as a natural interface for budget control. Empirical results show that AdaCtrl adapts reasoning length based on estimated difficulty, compared to the standard training baseline that also incorporates fine-tuning and RL, it yields performance improvements and simultaneously reduces response length by 10.06% and 12.14% on the more challenging AIME2024 and AIME2025 datasets, which require elaborate reasoning, and by 62.05% and 91.04% on the MATH500 and GSM8K datasets, where more concise responses are sufficient. Furthermore, AdaCtrl enables precise user control over the reasoning budget, allowing for tailored responses to meet specific needs.","url_abs":"https://arxiv.org/abs/2505.18822v1","url_pdf":"https://arxiv.org/pdf/2505.18822v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adactrl-towards-adaptive-and-controllable","repo_url":"https://github.com/joeying1019/adactrl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"gsm8k","task_name":"GSM8K"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2505.18822","atlas_url":"https://app.syntology.ai/?focus=2505.18822","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.18822"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/joeying1019/adactrl","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":3,"ran_fixture":1,"ran_draft_wrong":1,"unverified":10},"by_repo_kind":{"official":{"samples":15,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"de3d16d0873508ea","entry":"compute_ce_dpo_loss_rm","repo":"joeying1019/adactrl","repo_kind":"official","path":"recipe/prime/prime_core_algos.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/recipe/prime/prime_core_algos.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"de3d16d0873508ea"}},{"code_sha256_prefix":"b0af30a76b102578","entry":"compute_detach_dpo_loss_rm","repo":"joeying1019/adactrl","repo_kind":"official","path":"recipe/prime/prime_core_algos.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/recipe/prime/prime_core_algos.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b0af30a76b102578"}},{"code_sha256_prefix":"fe79c459a3e4cc64","entry":"get_huggingface_actor_config","repo":"joeying1019/adactrl","repo_kind":"official","path":"verl/utils/model.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/verl/utils/model.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fe79c459a3e4cc64"}},{"code_sha256_prefix":"21331a58f93375e4","entry":"union_tensor_dict","repo":"joeying1019/adactrl","repo_kind":"official","path":"verl/protocol.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/verl/protocol.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"21331a58f93375e4"}},{"code_sha256_prefix":"25f0ea3f460f6ce1","entry":"unpad_dataproto","repo":"joeying1019/adactrl","repo_kind":"official","path":"verl/protocol.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/verl/protocol.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"25f0ea3f460f6ce1"}},{"code_sha256_prefix":"875194b0e06d05e1","entry":"check_conversation_format","repo":"joeying1019/adactrl","repo_kind":"official","path":"utils/data_utils.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/utils/data_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"875194b0e06d05e1"}},{"code_sha256_prefix":"660713a8fa047736","entry":"get_generation_config","repo":"joeying1019/adactrl","repo_kind":"official","path":"verl/utils/model.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/verl/utils/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"660713a8fa047736"}},{"code_sha256_prefix":"af005d03c2088909","entry":"get_weight_loader","repo":"joeying1019/adactrl","repo_kind":"official","path":"verl/models/weight_loader_registry.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/verl/models/weight_loader_registry.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"af005d03c2088909"}},{"code_sha256_prefix":"914227ac881d05be","entry":"get_weight_saver","repo":"joeying1019/adactrl","repo_kind":"official","path":"verl/models/weight_loader_registry.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/verl/models/weight_loader_registry.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"914227ac881d05be"}},{"code_sha256_prefix":"2a180d3156997924","entry":"is_digit","repo":"joeying1019/adactrl","repo_kind":"official","path":"utils/openmathinst_utils.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/utils/openmathinst_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2a180d3156997924"}},{"code_sha256_prefix":"6b11d915b45fae0d","entry":"normalize","repo":"joeying1019/adactrl","repo_kind":"official","path":"utils/openmathinst_utils.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/utils/openmathinst_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6b11d915b45fae0d"}},{"code_sha256_prefix":"11d097e9be003804","entry":"normalize_answer_string","repo":"joeying1019/adactrl","repo_kind":"official","path":"utils/openmathinst_utils.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/utils/openmathinst_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"11d097e9be003804"}},{"code_sha256_prefix":"dd9ca8cdc6aa690d","entry":"read_json","repo":"joeying1019/adactrl","repo_kind":"official","path":"utils/data_utils.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/utils/data_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dd9ca8cdc6aa690d"}},{"code_sha256_prefix":"82f198e67affc0bb","entry":"read_jsonl","repo":"joeying1019/adactrl","repo_kind":"official","path":"utils/data_utils.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/utils/data_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"82f198e67affc0bb"}},{"code_sha256_prefix":"3b15e2ac7497c441","entry":"squeeze","repo":"joeying1019/adactrl","repo_kind":"official","path":"verl/utils/model.py","file_url":"https://github.com/joeying1019/adactrl/blob/HEAD/verl/utils/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3b15e2ac7497c441"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}