{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rl4co-an-extensive-reinforcement-learning-for","title":"RL4CO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmark","arxiv_id":"2306.17100","date":"2023-06-29","proceeding":null,"authors":["Federico Berto","Chuanbo Hua","Junyoung Park","Laurin Luttmann","Yining Ma","Fanchen Bu","Jiarui Wang","Haoran Ye","Minsu Kim","Sanghyeok Choi","Nayeli Gast Zepeda","André Hottung","Jianan Zhou","Jieyi Bi","Yu Hu","Fei Liu","Hyeonah Kim","Jiwoo Son","Haeyeon Kim","Davide Angioni","Wouter Kool","Zhiguang Cao","Qingfu Zhang","Joungho Kim","Jie Zhang","Kijung Shin","Cathy Wu","Sungsoo Ahn","Guojie Song","Changhyun Kwon","Kevin Tierney","Lin Xie","Jinkyoo Park"],"abstract":"Deep reinforcement learning (RL) has recently shown significant benefits in solving combinatorial optimization (CO) problems, reducing reliance on domain expertise, and improving computational efficiency. However, the field lacks a unified benchmark for easy development and standardized comparison of algorithms across diverse CO problems. To fill this gap, we introduce RL4CO, a unified and extensive benchmark with in-depth library coverage of 23 state-of-the-art methods and more than 20 CO problems. Built on efficient software libraries and best practices in implementation, RL4CO features modularized implementation and flexible configuration of diverse RL algorithms, neural network architectures, inference techniques, and environments. RL4CO allows researchers to seamlessly navigate existing successes and develop their unique designs, facilitating the entire research process by decoupling science from heavy engineering. We also provide extensive benchmark studies to inspire new insights and future work. RL4CO has attracted numerous researchers in the community and is open-sourced at https://github.com/ai4co/rl4co.","url_abs":"https://arxiv.org/abs/2306.17100v4","url_pdf":"https://arxiv.org/pdf/2306.17100v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rl4co-an-extensive-reinforcement-learning-for","repo_url":"https://github.com/ai4co/rl4co","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"rl4co-an-extensive-reinforcement-learning-for","repo_url":"https://github.com/pytorch/rl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":null},{"paper_slug":"rl4co-an-extensive-reinforcement-learning-for","repo_url":"https://github.com/facebookresearch/rl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"combinatorial-optimization","task_name":"Combinatorial Optimization"},{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"management","task_name":"Management"},{"task_slug":"navigate","task_name":"Navigate"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"traveling-salesman-problem","task_name":"Traveling Salesman Problem"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"am","method_name":"AM"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":null,"method_name":"Library"},{"method_slug":"pomo","method_name":"POMO"},{"method_slug":"pointer-net","method_name":"Pointer Network"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"sym-nco","method_name":"Sym-NCO"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.17100","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.17100"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ai4co/rl4co","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/rl","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pytorch/rl","reach":null}],"summary":{"unverified":16},"by_repo_kind":{"official":{"samples":13,"ran":0,"repositories":1},"listed":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"e899117c9df4359e","entry":"batchify","repo":"ai4co/rl4co","repo_kind":"official","path":"rl4co/utils/ops.py","file_url":"https://github.com/ai4co/rl4co/blob/HEAD/rl4co/utils/ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e899117c9df4359e"}},{"code_sha256_prefix":"4f2ccb8b68f3e715","entry":"clean_hydra_config","repo":"ai4co/rl4co","repo_kind":"official","path":"rl4co/utils/lightning.py","file_url":"https://github.com/ai4co/rl4co/blob/HEAD/rl4co/utils/lightning.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4f2ccb8b68f3e715"}},{"code_sha256_prefix":"c30fa333f4496ccc","entry":"create_optimizer","repo":"ai4co/rl4co","repo_kind":"official","path":"rl4co/utils/optim_helpers.py","file_url":"https://github.com/ai4co/rl4co/blob/HEAD/rl4co/utils/optim_helpers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c30fa333f4496ccc"}},{"code_sha256_prefix":"e57fec3fed7f7ac3","entry":"create_scheduler","repo":"ai4co/rl4co","repo_kind":"official","path":"rl4co/utils/optim_helpers.py","file_url":"https://github.com/ai4co/rl4co/blob/HEAD/rl4co/utils/optim_helpers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e57fec3fed7f7ac3"}},{"code_sha256_prefix":"7267ff9e6a317eec","entry":"fused_chunk_linear_attn_wrapper","repo":"ai4co/rl4co","repo_kind":"official","path":"rl4co/models/nn/flash_attention.py","file_url":"https://github.com/ai4co/rl4co/blob/HEAD/rl4co/models/nn/flash_attention.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7267ff9e6a317eec"}},{"code_sha256_prefix":"0fdfebdf5e82532b","entry":"gather_by_index","repo":"ai4co/rl4co","repo_kind":"official","path":"rl4co/utils/ops.py","file_url":"https://github.com/ai4co/rl4co/blob/HEAD/rl4co/utils/ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0fdfebdf5e82532b"}},{"code_sha256_prefix":"8a9f41865ce393ce","entry":"get_lightning_device","repo":"ai4co/rl4co","repo_kind":"official","path":"rl4co/utils/lightning.py","file_url":"https://github.com/ai4co/rl4co/blob/HEAD/rl4co/utils/lightning.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8a9f41865ce393ce"}},{"code_sha256_prefix":"a326212367641139","entry":"get_log_likelihood","repo":"ai4co/rl4co","repo_kind":"official","path":"rl4co/utils/decoding.py","file_url":"https://github.com/ai4co/rl4co/blob/HEAD/rl4co/utils/decoding.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a326212367641139"}},{"code_sha256_prefix":"1ea1f47aac5e2bda","entry":"pos_init_embedding","repo":"ai4co/rl4co","repo_kind":"official","path":"rl4co/models/nn/pos_embeddings.py","file_url":"https://github.com/ai4co/rl4co/blob/HEAD/rl4co/models/nn/pos_embeddings.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ea1f47aac5e2bda"}},{"code_sha256_prefix":"eb9333cd6b23461f","entry":"prod","repo":"facebookresearch/rl","repo_kind":"listed","path":"torchrl/_utils.py","file_url":"https://github.com/facebookresearch/rl/blob/HEAD/torchrl/_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"eb9333cd6b23461f"}},{"code_sha256_prefix":"cc7ca011efea6f39","entry":"remove_key","repo":"ai4co/rl4co","repo_kind":"official","path":"rl4co/utils/lightning.py","file_url":"https://github.com/ai4co/rl4co/blob/HEAD/rl4co/utils/lightning.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cc7ca011efea6f39"}},{"code_sha256_prefix":"59a1a205d6476e10","entry":"scaled_dot_product_attention_flash_attn","repo":"ai4co/rl4co","repo_kind":"official","path":"rl4co/models/nn/flash_attention.py","file_url":"https://github.com/ai4co/rl4co/blob/HEAD/rl4co/models/nn/flash_attention.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59a1a205d6476e10"}},{"code_sha256_prefix":"247b5d1c10709554","entry":"scaled_dot_product_attention_simple","repo":"ai4co/rl4co","repo_kind":"official","path":"rl4co/models/nn/attention.py","file_url":"https://github.com/ai4co/rl4co/blob/HEAD/rl4co/models/nn/attention.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"247b5d1c10709554"}},{"code_sha256_prefix":"f82fa5a8534a7ff2","entry":"seed_generator","repo":"facebookresearch/rl","repo_kind":"listed","path":"torchrl/_utils.py","file_url":"https://github.com/facebookresearch/rl/blob/HEAD/torchrl/_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"f82fa5a8534a7ff2"}},{"code_sha256_prefix":"f5efb02606aa8292","entry":"strtobool","repo":"facebookresearch/rl","repo_kind":"listed","path":"torchrl/_utils.py","file_url":"https://github.com/facebookresearch/rl/blob/HEAD/torchrl/_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"f5efb02606aa8292"}},{"code_sha256_prefix":"83c9a91ca5c59e38","entry":"unbatchify","repo":"ai4co/rl4co","repo_kind":"official","path":"rl4co/utils/ops.py","file_url":"https://github.com/ai4co/rl4co/blob/HEAD/rl4co/utils/ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"83c9a91ca5c59e38"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}