{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/self-improvement-for-neural-combinatorial","title":"Self-Improvement for Neural Combinatorial Optimization: Sample without Replacement, but Improvement","arxiv_id":"2403.15180","date":"2024-03-22","proceeding":null,"authors":["Jonathan Pirnay","Dominik G. Grimm"],"abstract":"Current methods for end-to-end constructive neural combinatorial optimization usually train a policy using behavior cloning from expert solutions or policy gradient methods from reinforcement learning. While behavior cloning is straightforward, it requires expensive expert solutions, and policy gradient methods are often computationally demanding and complex to fine-tune. In this work, we bridge the two and simplify the training process by sampling multiple solutions for random instances using the current model in each epoch and then selecting the best solution as an expert trajectory for supervised imitation learning. To achieve progressively improving solutions with minimal sampling, we introduce a method that combines round-wise Stochastic Beam Search with an update strategy derived from a provable policy improvement. This strategy refines the policy between rounds by utilizing the advantage of the sampled sequences with almost no computational overhead. We evaluate our approach on the Traveling Salesman Problem and the Capacitated Vehicle Routing Problem. The models trained with our method achieve comparable performance and generalization to those trained with expert data. Additionally, we apply our method to the Job Shop Scheduling Problem using a transformer-based architecture and outperform existing state-of-the-art methods by a wide margin.","url_abs":"https://arxiv.org/abs/2403.15180v2","url_pdf":"https://arxiv.org/pdf/2403.15180v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"self-improvement-for-neural-combinatorial","repo_url":"https://github.com/grimmlab/gumbeldore","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"combinatorial-optimization","task_name":"Combinatorial Optimization"},{"task_slug":"imitation-learning","task_name":"Imitation Learning"},{"task_slug":"job-shop-scheduling","task_name":"Job Shop Scheduling"},{"task_slug":"policy-gradient-methods","task_name":"Policy Gradient Methods"},{"task_slug":"scheduling","task_name":"Scheduling"},{"task_slug":"traveling-salesman-problem","task_name":"Traveling Salesman Problem"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.15180","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.15180"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/grimmlab/gumbeldore","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4a11e58d49369f6e","entry":"gumbel_log_survival","repo":"grimmlab/gumbeldore","repo_kind":"official","path":"core/incremental_sbs.py","file_url":"https://github.com/grimmlab/gumbeldore/blob/HEAD/core/incremental_sbs.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a11e58d49369f6e"}},{"code_sha256_prefix":"a0755f1307b73550","entry":"log_subtract","repo":"grimmlab/gumbeldore","repo_kind":"official","path":"core/incremental_sbs.py","file_url":"https://github.com/grimmlab/gumbeldore/blob/HEAD/core/incremental_sbs.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a0755f1307b73550"}},{"code_sha256_prefix":"39913f238caea7f8","entry":"sample_gumbels_with_maximum","repo":"grimmlab/gumbeldore","repo_kind":"official","path":"core/stochastic_beam_search.py","file_url":"https://github.com/grimmlab/gumbeldore/blob/HEAD/core/stochastic_beam_search.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"39913f238caea7f8"}},{"code_sha256_prefix":"b26c5b115681ee5a","entry":"softmax","repo":"grimmlab/gumbeldore","repo_kind":"official","path":"core/stochastic_beam_search.py","file_url":"https://github.com/grimmlab/gumbeldore/blob/HEAD/core/stochastic_beam_search.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b26c5b115681ee5a"}},{"code_sha256_prefix":"4cb5e196eec335b0","entry":"top_p_filtering","repo":"grimmlab/gumbeldore","repo_kind":"official","path":"core/stochastic_beam_search.py","file_url":"https://github.com/grimmlab/gumbeldore/blob/HEAD/core/stochastic_beam_search.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4cb5e196eec335b0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}