{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2602-12036","title":"Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Models","arxiv_id":"2602.12036","date":"2026-02-12","proceeding":null,"authors":["Xin Xu","Clive Bai","Kai Yang","Tianhao Chen","Yangkun Chen","Weijie Liu","Hao Chen","Yang Wang","Saiyong Yang","Can Yang"],"abstract":"Large-scale verifiable prompts underpin the success of Reinforcement Learning with Verifiable Rewards (RLVR), but they contain many uninformative examples and are costly to expand further. Recent studies focus on better exploiting limited training data by prioritizing hard prompts whose rollout pass rate is 0. However, easy prompts with a pass rate of 1 also become increasingly prevalent as training progresses, thereby reducing the effective data size. To mitigate this, we propose Composition-RL, a simple yet useful approach for better utilizing limited verifiable prompts targeting pass-rate-1 prompts. More specifically, Composition-RL automatically composes multiple problems into a new verifiable question and uses these compositional prompts for RL training. Extensive experiments across model sizes from 4B to 30B show that Composition-RL consistently improves reasoning capability over RL trained on the original dataset. Performance can be further boosted with a curriculum variant of Composition-RL that gradually increases compositional depth over training. Additionally, Composition-RL enables more effective cross-domain RL by composing prompts drawn from different domains. Codes, datasets, and models are available at https://github.com/XinXU-USTC/Composition-RL.","url_abs":"https://arxiv.org/abs/2602.12036","url_pdf":"https://arxiv.org/pdf/2602.12036","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2602.12036","atlas_url":"https://app.syntology.ai/?focus=2602.12036","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2602.12036"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/XinXU-USTC/Composition-RL","reach":null}],"summary":{"unverified":6},"by_repo_kind":{"found_in_text":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"7ab960c7f4b87508","entry":"gen_response","repo":"XinXU-USTC/Composition-RL","repo_kind":"found_in_text","path":"NCSP/src/mcts_with_completion.py","file_url":"https://github.com/XinXU-USTC/Composition-RL/blob/HEAD/NCSP/src/mcts_with_completion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7ab960c7f4b87508"}},{"code_sha256_prefix":"acf240079d434746","entry":"get_weight_loader","repo":"XinXU-USTC/Composition-RL","repo_kind":"found_in_text","path":"verl/models/weight_loader_registry.py","file_url":"https://github.com/XinXU-USTC/Composition-RL/blob/HEAD/verl/models/weight_loader_registry.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"acf240079d434746"}},{"code_sha256_prefix":"cd6b1e0d6afb2762","entry":"get_weight_saver","repo":"XinXU-USTC/Composition-RL","repo_kind":"found_in_text","path":"verl/models/weight_loader_registry.py","file_url":"https://github.com/XinXU-USTC/Composition-RL/blob/HEAD/verl/models/weight_loader_registry.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cd6b1e0d6afb2762"}},{"code_sha256_prefix":"ab70e58f0c66b649","entry":"is_terminal","repo":"XinXU-USTC/Composition-RL","repo_kind":"found_in_text","path":"NCSP/src/mcts_with_completion.py","file_url":"https://github.com/XinXU-USTC/Composition-RL/blob/HEAD/NCSP/src/mcts_with_completion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ab70e58f0c66b649"}},{"code_sha256_prefix":"90e3fe05833d54a9","entry":"run_python","repo":"XinXU-USTC/Composition-RL","repo_kind":"found_in_text","path":"NCSP/src/utils.py","file_url":"https://github.com/XinXU-USTC/Composition-RL/blob/HEAD/NCSP/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"90e3fe05833d54a9"}},{"code_sha256_prefix":"797d7680ab975c2c","entry":"weighted_random_choice","repo":"XinXU-USTC/Composition-RL","repo_kind":"found_in_text","path":"NCSP/src/utils.py","file_url":"https://github.com/XinXU-USTC/Composition-RL/blob/HEAD/NCSP/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"797d7680ab975c2c"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}