{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/parametrized-deep-q-networks-learning","title":"Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space","arxiv_id":"1810.06394","date":"2018-10-10","proceeding":null,"authors":["Jiechao Xiong","Qing Wang","Zhuoran Yang","Peng Sun","Lei Han","Yang Zheng","Haobo Fu","Tong Zhang","Ji Liu","Han Liu"],"abstract":"Most existing deep reinforcement learning (DRL) frameworks consider either\ndiscrete action space or continuous action space solely. Motivated by\napplications in computer games, we consider the scenario with\ndiscrete-continuous hybrid action space. To handle hybrid action space,\nprevious works either approximate the hybrid space by discretization, or relax\nit into a continuous set. In this paper, we propose a parametrized deep\nQ-network (P- DQN) framework for the hybrid action space without approximation\nor relaxation. Our algorithm combines the spirits of both DQN (dealing with\ndiscrete action space) and DDPG (dealing with continuous action space) by\nseamlessly integrating them. Empirical results on a simulation example, scoring\na goal in simulated RoboCup soccer and the solo mode in game King of Glory\n(KOG) validate the efficiency and effectiveness of our method.","url_abs":"http://arxiv.org/abs/1810.06394v1","url_pdf":"http://arxiv.org/pdf/1810.06394v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"parametrized-deep-q-networks-learning","repo_url":"https://github.com/SeungeonBaek/hybrid-agents-test","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"parametrized-deep-q-networks-learning","repo_url":"https://github.com/cycraig/MP-DQN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"parametrized-deep-q-networks-learning","repo_url":"https://github.com/lollcat/RL-Process-Design","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"parametrized-deep-q-networks-learning","repo_url":"https://github.com/opendilab/DI-engine","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"parametrized-deep-q-networks-learning","repo_url":"https://github.com/thomashirtz/gym-hybrid","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"ddpg","method_name":"DDPG"},{"method_slug":"dqn","method_name":"DQN"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"experience-replay","method_name":"Experience Replay"},{"method_slug":"q-learning","method_name":"Q-Learning"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1810.06394","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1810.06394"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cycraig/MP-DQN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SeungeonBaek/hybrid-agents-test","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lollcat/RL-Process-Design","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thomashirtz/gym-hybrid","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/opendilab/DI-engine","reach":null}],"summary":{"ran_honours":1,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"listed":{"samples":3,"ran":2,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"42cfe92e5593c56f","entry":"evaluate","repo":"cycraig/MP-DQN","repo_kind":"listed","path":"run_platform_pdqn.py","file_url":"https://github.com/cycraig/MP-DQN/blob/HEAD/run_platform_pdqn.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"42cfe92e5593c56f"}},{"code_sha256_prefix":"7277746b5fb06a33","entry":"pad_action","repo":"cycraig/MP-DQN","repo_kind":"listed","path":"run_platform_pdqn.py","file_url":"https://github.com/cycraig/MP-DQN/blob/HEAD/run_platform_pdqn.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7277746b5fb06a33"}},{"code_sha256_prefix":"92ef5d0d5bc7717e","entry":"env_agent_config","repo":"SeungeonBaek/hybrid-agents-test","repo_kind":"listed","path":"agent_env_config.py","file_url":"https://github.com/SeungeonBaek/hybrid-agents-test/blob/HEAD/agent_env_config.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"92ef5d0d5bc7717e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}