{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/action-branching-architectures-for-deep","title":"Action Branching Architectures for Deep Reinforcement Learning","arxiv_id":"1711.08946","date":"2017-11-24","proceeding":null,"authors":["Arash Tavakoli","Fabio Pardo","Petar Kormushev"],"abstract":"Discrete-action algorithms have been central to numerous recent successes of\ndeep reinforcement learning. However, applying these algorithms to\nhigh-dimensional action tasks requires tackling the combinatorial increase of\nthe number of possible actions with the number of action dimensions. This\nproblem is further exacerbated for continuous-action tasks that require fine\ncontrol of actions via discretization. In this paper, we propose a novel neural\narchitecture featuring a shared decision module followed by several network\nbranches, one for each action dimension. This approach achieves a linear\nincrease of the number of network outputs with the number of degrees of freedom\nby allowing a level of independence for each individual action dimension. To\nillustrate the approach, we present a novel agent, called Branching Dueling\nQ-Network (BDQ), as a branching variant of the Dueling Double Deep Q-Network\n(Dueling DDQN). We evaluate the performance of our agent on a set of\nchallenging continuous control tasks. The empirical results show that the\nproposed agent scales gracefully to environments with increasing action\ndimensionality and indicate the significance of the shared decision module in\ncoordination of the distributed action branches. Furthermore, we show that the\nproposed agent performs competitively against a state-of-the-art continuous\ncontrol algorithm, Deep Deterministic Policy Gradient (DDPG).","url_abs":"http://arxiv.org/abs/1711.08946v2","url_pdf":"http://arxiv.org/pdf/1711.08946v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"action-branching-architectures-for-deep","repo_url":"https://github.com/atavakol/action-branching-agents","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"action-branching-architectures-for-deep","repo_url":"https://github.com/BFAnas/BranchingDQN_keras","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"action-branching-architectures-for-deep","repo_url":"https://github.com/MoMe36/BranchingDQN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"action-branching-architectures-for-deep","repo_url":"https://github.com/atavakol/action-hypergraph-networks","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"action-branching-architectures-for-deep","repo_url":"https://github.com/bastienvanderplaetse/action-branching-dqn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"continuous-control","task_name":"Continuous Control"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"general-reinforcement-learning","task_name":"General Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"continuous-control","task_name":"continuous-control"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1711.08946","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}