{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/combinatorial-optimization-by-graph-pointer","title":"Combinatorial Optimization by Graph Pointer Networks and Hierarchical Reinforcement Learning","arxiv_id":"1911.04936","date":"2019-11-12","proceeding":null,"authors":["Qiang Ma","Suwen Ge","Danyang He","Darshan Thaker","Iddo Drori"],"abstract":"In this work, we introduce Graph Pointer Networks (GPNs) trained using reinforcement learning (RL) for tackling the traveling salesman problem (TSP). GPNs build upon Pointer Networks by introducing a graph embedding layer on the input, which captures relationships between nodes. Furthermore, to approximate solutions to constrained combinatorial optimization problems such as the TSP with time windows, we train hierarchical GPNs (HGPNs) using RL, which learns a hierarchical policy to find an optimal city permutation under constraints. Each layer of the hierarchy is designed with a separate reward function, resulting in stable training. Our results demonstrate that GPNs trained on small-scale TSP50/100 problems generalize well to larger-scale TSP500/1000 problems, with shorter tour lengths and faster computational times. We verify that for constrained TSP problems such as the TSP with time windows, the feasible solutions found via hierarchical RL training outperform previous baselines. In the spirit of reproducible research we make our data, models, and code publicly available.","url_abs":"https://arxiv.org/abs/1911.04936v1","url_pdf":"https://arxiv.org/pdf/1911.04936v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"combinatorial-optimization-by-graph-pointer","repo_url":"https://github.com/qiang-ma/graph-pointer-network","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"combinatorial-optimization-by-graph-pointer","repo_url":"https://github.com/danifuertes/top_transformer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"combinatorial-optimization","task_name":"Combinatorial Optimization"},{"task_slug":"graph-embedding","task_name":"Graph Embedding"},{"task_slug":"hierarchical-reinforcement-learning","task_name":"Hierarchical Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"traveling-salesman-problem","task_name":"Traveling Salesman Problem"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1911.04936","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1911.04936"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/danifuertes/top_transformer","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qiang-ma/graph-pointer-network","reach":{"status":"ok"}}],"summary":{"unverified":9},"by_repo_kind":{"listed":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"53acaabe82784082","entry":"calc_op_length","repo":"danifuertes/top_transformer","repo_kind":"listed","path":"problems/top/eval_baselines.py","file_url":"https://github.com/danifuertes/top_transformer/blob/HEAD/problems/top/eval_baselines.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"53acaabe82784082"}},{"code_sha256_prefix":"996e8c67ffbd318b","entry":"calc_op_total","repo":"danifuertes/top_transformer","repo_kind":"listed","path":"problems/top/eval_baselines.py","file_url":"https://github.com/danifuertes/top_transformer/blob/HEAD/problems/top/eval_baselines.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"996e8c67ffbd318b"}},{"code_sha256_prefix":"70bb4fdd552dec0b","entry":"calculate_distance","repo":"danifuertes/top_transformer","repo_kind":"listed","path":"problems/top/compass.py","file_url":"https://github.com/danifuertes/top_transformer/blob/HEAD/problems/top/compass.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70bb4fdd552dec0b"}},{"code_sha256_prefix":"74978bcc80c74800","entry":"ci","repo":"danifuertes/top_transformer","repo_kind":"listed","path":"problems/top/eval_baselines.py","file_url":"https://github.com/danifuertes/top_transformer/blob/HEAD/problems/top/eval_baselines.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"74978bcc80c74800"}},{"code_sha256_prefix":"1a32218019e75cc7","entry":"crossover_","repo":"danifuertes/top_transformer","repo_kind":"listed","path":"problems/top/opga.py","file_url":"https://github.com/danifuertes/top_transformer/blob/HEAD/problems/top/opga.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a32218019e75cc7"}},{"code_sha256_prefix":"548e1aeeaf089fab","entry":"fitness_","repo":"danifuertes/top_transformer","repo_kind":"listed","path":"problems/top/opga.py","file_url":"https://github.com/danifuertes/top_transformer/blob/HEAD/problems/top/opga.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"548e1aeeaf089fab"}},{"code_sha256_prefix":"0953b2cbca71dea4","entry":"generate_instance","repo":"danifuertes/top_transformer","repo_kind":"listed","path":"problems/top/problem_top.py","file_url":"https://github.com/danifuertes/top_transformer/blob/HEAD/problems/top/problem_top.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0953b2cbca71dea4"}},{"code_sha256_prefix":"5abb836225370fd2","entry":"generate_top_data","repo":"danifuertes/top_transformer","repo_kind":"listed","path":"generate_data.py","file_url":"https://github.com/danifuertes/top_transformer/blob/HEAD/generate_data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5abb836225370fd2"}},{"code_sha256_prefix":"7c32a4ba11e30a12","entry":"top_compass","repo":"danifuertes/top_transformer","repo_kind":"listed","path":"problems/top/compass.py","file_url":"https://github.com/danifuertes/top_transformer/blob/HEAD/problems/top/compass.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7c32a4ba11e30a12"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}