{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multi-agent-deep-reinforcement-learning-for-2","title":"Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control","arxiv_id":"1903.04527","date":"2019-03-11","proceeding":null,"authors":["Tianshu Chu","Jie Wang","Lara Codecà","Zhaojian Li"],"abstract":"Reinforcement learning (RL) is a promising data-driven approach for adaptive\ntraffic signal control (ATSC) in complex urban traffic networks, and deep\nneural networks further enhance its learning power. However, centralized RL is\ninfeasible for large-scale ATSC due to the extremely high dimension of the\njoint action space. Multi-agent RL (MARL) overcomes the scalability issue by\ndistributing the global control to each local RL agent, but it introduces new\nchallenges: now the environment becomes partially observable from the viewpoint\nof each local agent due to limited communication among agents. Most existing\nstudies in MARL focus on designing efficient communication and coordination\namong traditional Q-learning agents. This paper presents, for the first time, a\nfully scalable and decentralized MARL algorithm for the state-of-the-art deep\nRL agent: advantage actor critic (A2C), within the context of ATSC. In\nparticular, two methods are proposed to stabilize the learning procedure, by\nimproving the observability and reducing the learning difficulty of each local\nagent. The proposed multi-agent A2C is compared against independent A2C and\nindependent Q-learning algorithms, in both a large synthetic traffic grid and a\nlarge real-world traffic network of Monaco city, under simulated peak-hour\ntraffic dynamics. Results demonstrate its optimality, robustness, and sample\nefficiency over other state-of-the-art decentralized MARL algorithms.","url_abs":"http://arxiv.org/abs/1903.04527v1","url_pdf":"http://arxiv.org/pdf/1903.04527v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multi-agent-deep-reinforcement-learning-for-2","repo_url":"https://github.com/cts198859/deeprl_signal_control","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"q-learning","task_name":"Q-Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"traffic-signal-control","task_name":"Traffic Signal Control"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"a2c","method_name":"A2C"},{"method_slug":"q-learning","method_name":"Q-Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1903.04527","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1903.04527"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cts198859/deeprl_signal_control","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"unverified":8},"by_repo_kind":{"official":{"samples":9,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6583a667fee3e221","entry":"ortho_init","repo":"cts198859/deeprl_signal_control","repo_kind":"official","path":"agents/utils.py","file_url":"https://github.com/cts198859/deeprl_signal_control/blob/HEAD/agents/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6583a667fee3e221"}},{"code_sha256_prefix":"a774e08f7631887e","entry":"check_dir","repo":"cts198859/deeprl_signal_control","repo_kind":"official","path":"utils.py","file_url":"https://github.com/cts198859/deeprl_signal_control/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a774e08f7631887e"}},{"code_sha256_prefix":"3d1a6d68723115b4","entry":"conv","repo":"cts198859/deeprl_signal_control","repo_kind":"official","path":"agents/utils.py","file_url":"https://github.com/cts198859/deeprl_signal_control/blob/HEAD/agents/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d1a6d68723115b4"}},{"code_sha256_prefix":"bd2260a92d6fcfef","entry":"create_multiplexer","repo":"cts198859/deeprl_signal_control","repo_kind":"official","path":"extract_tensorboard.py","file_url":"https://github.com/cts198859/deeprl_signal_control/blob/HEAD/extract_tensorboard.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bd2260a92d6fcfef"}},{"code_sha256_prefix":"2a1c3fc637c91a31","entry":"extract_scalar","repo":"cts198859/deeprl_signal_control","repo_kind":"official","path":"extract_tensorboard.py","file_url":"https://github.com/cts198859/deeprl_signal_control/blob/HEAD/extract_tensorboard.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a1c3fc637c91a31"}},{"code_sha256_prefix":"b2fb59622dad3ce4","entry":"find_file","repo":"cts198859/deeprl_signal_control","repo_kind":"official","path":"utils.py","file_url":"https://github.com/cts198859/deeprl_signal_control/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b2fb59622dad3ce4"}},{"code_sha256_prefix":"eeb0e1e913b454fe","entry":"get_event_log","repo":"cts198859/deeprl_signal_control","repo_kind":"official","path":"extract_tensorboard.py","file_url":"https://github.com/cts198859/deeprl_signal_control/blob/HEAD/extract_tensorboard.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eeb0e1e913b454fe"}},{"code_sha256_prefix":"636e940d9b81af2a","entry":"init_dir","repo":"cts198859/deeprl_signal_control","repo_kind":"official","path":"utils.py","file_url":"https://github.com/cts198859/deeprl_signal_control/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"636e940d9b81af2a"}},{"code_sha256_prefix":"5af19b29d5176250","entry":"norm_init","repo":"cts198859/deeprl_signal_control","repo_kind":"official","path":"agents/utils.py","file_url":"https://github.com/cts198859/deeprl_signal_control/blob/HEAD/agents/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5af19b29d5176250"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}