{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/qtran-learning-to-factorize-with","title":"QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning","arxiv_id":"1905.05408","date":"2019-05-14","proceeding":null,"authors":["Kyunghwan Son","Daewoo Kim","Wan Ju Kang","David Earl Hostallero","Yung Yi"],"abstract":"We explore value-based solutions for multi-agent reinforcement learning (MARL) tasks in the centralized training with decentralized execution (CTDE) regime popularized recently. However, VDN and QMIX are representative examples that use the idea of factorization of the joint action-value function into individual ones for decentralized execution. VDN and QMIX address only a fraction of factorizable MARL tasks due to their structural constraint in factorization such as additivity and monotonicity. In this paper, we propose a new factorization method for MARL, QTRAN, which is free from such structural constraints and takes on a new approach to transforming the original joint action-value function into an easily factorizable one, with the same optimal actions. QTRAN guarantees more general factorization than VDN or QMIX, thus covering a much wider class of MARL tasks than does previous methods. Our experiments for the tasks of multi-domain Gaussian-squeeze and modified predator-prey demonstrate QTRAN's superior performance with especially larger margins in games whose payoffs penalize non-cooperative behavior more aggressively.","url_abs":"https://arxiv.org/abs/1905.05408v1","url_pdf":"https://arxiv.org/pdf/1905.05408v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"qtran-learning-to-factorize-with","repo_url":"https://github.com/Sonkyunghwan/QTRAN","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"qtran-learning-to-factorize-with","repo_url":"https://github.com/hhhusiyi-monash/UPDeT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"qtran-learning-to-factorize-with","repo_url":"https://github.com/jugg1er/air","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"qtran-learning-to-factorize-with","repo_url":"https://github.com/opendilab/DI-engine/blob/main/ding/policy/qtran.py","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"multi-agent-reinforcement-learning","task_name":"Multi-agent Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"smac-1","task_name":"SMAC+"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/smac-on-smac-def-armored-parallel","task":"SMAC+","dataset":"Def_Armored_parallel","model":"QTRAN","rank_in_archive_order":4,"of":10,"metrics":{"Median Win Rate":"5.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-armored-sequential","task":"SMAC+","dataset":"Def_Armored_sequential","model":"QTRAN","rank_in_archive_order":3,"of":11,"metrics":{"Median Win Rate":"93.8"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-infantry-parallel","task":"SMAC+","dataset":"Def_Infantry_parallel","model":"QTRAN","rank_in_archive_order":1,"of":10,"metrics":{"Median Win Rate":"100.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-infantry-sequential","task":"SMAC+","dataset":"Def_Infantry_sequential","model":"QTRAN","rank_in_archive_order":3,"of":11,"metrics":{"Median Win Rate":"100"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-outnumbered-parallel","task":"SMAC+","dataset":"Def_Outnumbered_parallel","model":"QTRAN","rank_in_archive_order":7,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-outnumbered-sequential","task":"SMAC+","dataset":"Def_Outnumbered_sequential","model":"QTRAN","rank_in_archive_order":3,"of":11,"metrics":{"Median Win Rate":"81.3"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-complicated-parallel","task":"SMAC+","dataset":"Off_Complicated_parallel","model":"QTRAN","rank_in_archive_order":6,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-distant-parallel","task":"SMAC+","dataset":"Off_Distant_parallel","model":"QTRAN","rank_in_archive_order":6,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-hard-parallel","task":"SMAC+","dataset":"Off_Hard_parallel","model":"QTRAN","rank_in_archive_order":6,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-near-parallel","task":"SMAC+","dataset":"Off_Near_parallel","model":"QTRAN","rank_in_archive_order":7,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-superhard-parallel","task":"SMAC+","dataset":"Off_Superhard_parallel","model":"QTRAN","rank_in_archive_order":4,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1905.05408","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1905.05408"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/opendilab/DI-engine/blob/main/ding/policy/qtran.py","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hhhusiyi-monash/UPDeT","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Sonkyunghwan/QTRAN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jugg1er/air","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"ec3f64501ca07cda","entry":"make_env","repo":"Sonkyunghwan/QTRAN","repo_kind":"official","path":"Others/make_env.py","file_url":"https://github.com/Sonkyunghwan/QTRAN/blob/HEAD/Others/make_env.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ec3f64501ca07cda"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}