{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/qplex-duplex-dueling-multi-agent-q-learning","title":"QPLEX: Duplex Dueling Multi-Agent Q-Learning","arxiv_id":"2008.01062","date":"2020-08-03","proceeding":"ICLR 2021 1","authors":["Jianhao Wang","Zhizhou Ren","Terry Liu","Yang Yu","Chongjie Zhang"],"abstract":"We explore value-based multi-agent reinforcement learning (MARL) in the popular paradigm of centralized training with decentralized execution (CTDE). CTDE has an important concept, Individual-Global-Max (IGM) principle, which requires the consistency between joint and local action selections to support efficient local decision-making. However, in order to achieve scalability, existing MARL methods either limit representation expressiveness of their value function classes or relax the IGM consistency, which may suffer from instability risk or may not perform well in complex domains. This paper presents a novel MARL approach, called duPLEX dueling multi-agent Q-learning (QPLEX), which takes a duplex dueling network architecture to factorize the joint value function. This duplex dueling structure encodes the IGM principle into the neural network architecture and thus enables efficient value function learning. Theoretical analysis shows that QPLEX achieves a complete IGM function class. Empirical experiments on StarCraft II micromanagement tasks demonstrate that QPLEX significantly outperforms state-of-the-art baselines in both online and offline data collection settings, and also reveal that QPLEX achieves high sample efficiency and can benefit from offline datasets without additional online exploration.","url_abs":"https://arxiv.org/abs/2008.01062v3","url_pdf":"https://arxiv.org/pdf/2008.01062v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"qplex-duplex-dueling-multi-agent-q-learning","repo_url":"https://github.com/wjh720/QPLEX","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"qplex-duplex-dueling-multi-agent-q-learning","repo_url":"https://github.com/cathyhxh/ctds","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"qplex-duplex-dueling-multi-agent-q-learning","repo_url":"https://github.com/hyunghona/emu","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"qplex-duplex-dueling-multi-agent-q-learning","repo_url":"https://github.com/oxwhirl/wqmix","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"qplex-duplex-dueling-multi-agent-q-learning","repo_url":"https://github.com/somnathhazra/uncertainties_marl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"qplex-duplex-dueling-multi-agent-q-learning","repo_url":"https://github.com/umersheikh846/WQMIX","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"multi-agent-reinforcement-learning","task_name":"Multi-agent Reinforcement Learning"},{"task_slug":"q-learning","task_name":"Q-Learning"},{"task_slug":"starcraft","task_name":"Starcraft"},{"task_slug":"starcraft-ii","task_name":"Starcraft II"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"double-q-learning","method_name":"Double Q-learning"},{"method_slug":"dueling-network","method_name":"Dueling Network"},{"method_slug":"q-learning","method_name":"Q-Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2008.01062","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2008.01062"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wjh720/QPLEX","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/somnathhazra/uncertainties_marl","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hyunghona/emu","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/umersheikh846/WQMIX","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/oxwhirl/wqmix","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cathyhxh/ctds","reach":{"status":"ok"}}],"summary":{"ran":2,"unverified":2},"by_repo_kind":{"listed":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6008b0eb92d8067c","entry":"categorical_entropy","repo":"somnathhazra/uncertainties_marl","repo_kind":"listed","path":"src/components/action_selectors.py","file_url":"https://github.com/somnathhazra/uncertainties_marl/blob/HEAD/src/components/action_selectors.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6008b0eb92d8067c"}},{"code_sha256_prefix":"1da3ad2020291e67","entry":"multinomial_entropy","repo":"somnathhazra/uncertainties_marl","repo_kind":"listed","path":"src/components/action_selectors.py","file_url":"https://github.com/somnathhazra/uncertainties_marl/blob/HEAD/src/components/action_selectors.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1da3ad2020291e67"}},{"code_sha256_prefix":"012625dd7922e3c6","entry":"args_sanity_check","repo":"somnathhazra/uncertainties_marl","repo_kind":"listed","path":"src/run/run_dist.py","file_url":"https://github.com/somnathhazra/uncertainties_marl/blob/HEAD/src/run/run_dist.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"012625dd7922e3c6"}},{"code_sha256_prefix":"c5e0750dc161d1e0","entry":"calculate_target_q","repo":"somnathhazra/uncertainties_marl","repo_kind":"listed","path":"src/learners/iqn_learner_dist.py","file_url":"https://github.com/somnathhazra/uncertainties_marl/blob/HEAD/src/learners/iqn_learner_dist.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c5e0750dc161d1e0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}