{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dfac-framework-factorizing-the-value-function","title":"DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learning","arxiv_id":"2102.07936","date":"2021-02-16","proceeding":null,"authors":["Wei-Fang Sun","Cheng-Kuang Lee","Chun-Yi Lee"],"abstract":"In fully cooperative multi-agent reinforcement learning (MARL) settings, the environments are highly stochastic due to the partial observability of each agent and the continuously changing policies of the other agents. To address the above issues, we integrate distributional RL and value function factorization methods by proposing a Distributional Value Function Factorization (DFAC) framework to generalize expected value function factorization methods to their DFAC variants. DFAC extends the individual utility functions from deterministic variables to random variables, and models the quantile function of the total return as a quantile mixture. To validate DFAC, we demonstrate DFAC's ability to factorize a simple two-step matrix game with stochastic rewards and perform experiments on all Super Hard tasks of StarCraft Multi-Agent Challenge, showing that DFAC is able to outperform expected value function factorization baselines.","url_abs":"https://arxiv.org/abs/2102.07936v2","url_pdf":"https://arxiv.org/pdf/2102.07936v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dfac-framework-factorizing-the-value-function","repo_url":"https://github.com/j3soon/dfac","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"multi-agent-reinforcement-learning","task_name":"Multi-agent Reinforcement Learning"},{"task_slug":"q-learning","task_name":"Q-Learning"},{"task_slug":"smac","task_name":"SMAC"},{"task_slug":"smac-1","task_name":"SMAC+"},{"task_slug":"starcraft","task_name":"Starcraft"}],"methods":[{"method_slug":"q-learning","method_name":"Q-Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/smac-on-smac-27m-vs-30m","task":"SMAC","dataset":"SMAC 27m_vs_30m","model":"DDN","rank_in_archive_order":1,"of":11,"metrics":{"Average Score":"19.71","Median Win Rate":"91.48"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-27m-vs-30m","task":"SMAC","dataset":"SMAC 27m_vs_30m","model":"DMIX","rank_in_archive_order":3,"of":11,"metrics":{"Average Score":"19.43","Median Win Rate":"85.45"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-27m-vs-30m","task":"SMAC","dataset":"SMAC 27m_vs_30m","model":"QMIX","rank_in_archive_order":4,"of":11,"metrics":{"Average Score":"19.41","Median Win Rate":"84.77"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-27m-vs-30m","task":"SMAC","dataset":"SMAC 27m_vs_30m","model":"VDN","rank_in_archive_order":6,"of":11,"metrics":{"Average Score":"18.45","Median Win Rate":"63.12"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-27m-vs-30m","task":"SMAC","dataset":"SMAC 27m_vs_30m","model":"DIQL","rank_in_archive_order":9,"of":11,"metrics":{"Average Score":"14.45","Median Win Rate":"6.02"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-27m-vs-30m","task":"SMAC","dataset":"SMAC 27m_vs_30m","model":"IQL","rank_in_archive_order":10,"of":11,"metrics":{"Average Score":"14.01","Median Win Rate":"2.27"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-3s5z-vs-3s6z-1","task":"SMAC","dataset":"SMAC 3s5z_vs_3s6z","model":"DDN","rank_in_archive_order":2,"of":13,"metrics":{"Average Score":"20.94","Median Win Rate":"94.03"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-3s5z-vs-3s6z-1","task":"SMAC","dataset":"SMAC 3s5z_vs_3s6z","model":"DMIX","rank_in_archive_order":3,"of":13,"metrics":{"Average Score":"19.7","Median Win Rate":"91.08"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-3s5z-vs-3s6z-1","task":"SMAC","dataset":"SMAC 3s5z_vs_3s6z","model":"VDN","rank_in_archive_order":5,"of":13,"metrics":{"Average Score":"19.75","Median Win Rate":"89.2"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-3s5z-vs-3s6z-1","task":"SMAC","dataset":"SMAC 3s5z_vs_3s6z","model":"QMIX","rank_in_archive_order":7,"of":13,"metrics":{"Average Score":"20.16","Median Win Rate":"67.22"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-3s5z-vs-3s6z-1","task":"SMAC","dataset":"SMAC 3s5z_vs_3s6z","model":"DIQL","rank_in_archive_order":8,"of":13,"metrics":{"Average Score":"17.52","Median Win Rate":"62.22"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-3s5z-vs-3s6z-1","task":"SMAC","dataset":"SMAC 3s5z_vs_3s6z","model":"IQL","rank_in_archive_order":9,"of":13,"metrics":{"Average Score":"16.54","Median Win Rate":"29.83"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-6h-vs-8z-1","task":"SMAC","dataset":"SMAC 6h_vs_8z","model":"DDN","rank_in_archive_order":2,"of":14,"metrics":{"Average Score":"19.4","Median Win Rate":"83.92"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-6h-vs-8z-1","task":"SMAC","dataset":"SMAC 6h_vs_8z","model":"DMIX","rank_in_archive_order":3,"of":14,"metrics":{"Average Score":"17.14","Median Win Rate":"49.43"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-6h-vs-8z-1","task":"SMAC","dataset":"SMAC 6h_vs_8z","model":"QMIX","rank_in_archive_order":5,"of":14,"metrics":{"Average Score":"14.37","Median Win Rate":"12.78"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-6h-vs-8z-1","task":"SMAC","dataset":"SMAC 6h_vs_8z","model":"VDN","rank_in_archive_order":8,"of":14,"metrics":{"Average Score":"15.41","Median Win Rate":"0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-6h-vs-8z-1","task":"SMAC","dataset":"SMAC 6h_vs_8z","model":"DIQL","rank_in_archive_order":9,"of":14,"metrics":{"Average Score":"14.94","Median Win Rate":"0.00"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-6h-vs-8z-1","task":"SMAC","dataset":"SMAC 6h_vs_8z","model":"IQL","rank_in_archive_order":10,"of":14,"metrics":{"Average Score":"13.78","Median Win Rate":"0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-mmm2-1","task":"SMAC","dataset":"SMAC MMM2","model":"DDN","rank_in_archive_order":2,"of":14,"metrics":{"Average Score":"20.9","Median Win Rate":"97.22"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-mmm2-1","task":"SMAC","dataset":"SMAC MMM2","model":"DMIX","rank_in_archive_order":5,"of":14,"metrics":{"Average Score":"19.87","Median Win Rate":"95.11"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-mmm2-1","task":"SMAC","dataset":"SMAC MMM2","model":"QMIX","rank_in_archive_order":6,"of":14,"metrics":{"Average Score":"19.42","Median Win Rate":"92.44"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-mmm2-1","task":"SMAC","dataset":"SMAC MMM2","model":"VDN","rank_in_archive_order":7,"of":14,"metrics":{"Average Score":"19.36","Median Win Rate":"89.2"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-mmm2-1","task":"SMAC","dataset":"SMAC MMM2","model":"DIQL","rank_in_archive_order":8,"of":14,"metrics":{"Average Score":"19.21","Median Win Rate":"85.23"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-mmm2-1","task":"SMAC","dataset":"SMAC MMM2","model":"IQL","rank_in_archive_order":11,"of":14,"metrics":{"Average Score":"17.5","Median Win Rate":"68.92"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-corridor","task":"SMAC","dataset":"SMAC corridor","model":"DDN","rank_in_archive_order":2,"of":13,"metrics":{"Average Score":"20","Median Win Rate":"95.4"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-corridor","task":"SMAC","dataset":"SMAC corridor","model":"DIQL","rank_in_archive_order":3,"of":13,"metrics":{"Average Score":"19.68","Median Win Rate":"91.62"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-corridor","task":"SMAC","dataset":"SMAC corridor","model":"DMIX","rank_in_archive_order":4,"of":13,"metrics":{"Average Score":"19.66","Median Win Rate":"90.45"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-corridor","task":"SMAC","dataset":"SMAC corridor","model":"VDN","rank_in_archive_order":5,"of":13,"metrics":{"Average Score":"19.47","Median Win Rate":"85.34"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-corridor","task":"SMAC","dataset":"SMAC corridor","model":"IQL","rank_in_archive_order":6,"of":13,"metrics":{"Average Score":"19.42","Median Win Rate":"84.87"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-corridor","task":"SMAC","dataset":"SMAC corridor","model":"QMIX","rank_in_archive_order":9,"of":13,"metrics":{"Average Score":"15.07","Median Win Rate":"37.61"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-armored-parallel","task":"SMAC+","dataset":"Def_Armored_parallel","model":"DMIX","rank_in_archive_order":1,"of":10,"metrics":{"Median Win Rate":"90.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-armored-parallel","task":"SMAC+","dataset":"Def_Armored_parallel","model":"DDN","rank_in_archive_order":9,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-armored-parallel","task":"SMAC+","dataset":"Def_Armored_parallel","model":"DIQL","rank_in_archive_order":10,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-armored-sequential","task":"SMAC+","dataset":"Def_Armored_sequential","model":"DMIX","rank_in_archive_order":5,"of":11,"metrics":{"Median Win Rate":"81.3"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-armored-sequential","task":"SMAC+","dataset":"Def_Armored_sequential","model":"DDN","rank_in_archive_order":6,"of":11,"metrics":{"Median Win Rate":"71.9"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-armored-sequential","task":"SMAC+","dataset":"Def_Armored_sequential","model":"DIQL","rank_in_archive_order":7,"of":11,"metrics":{"Median Win Rate":"53.1"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-infantry-parallel","task":"SMAC+","dataset":"Def_Infantry_parallel","model":"DMIX","rank_in_archive_order":5,"of":10,"metrics":{"Median Win Rate":"90.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-infantry-parallel","task":"SMAC+","dataset":"Def_Infantry_parallel","model":"DDN","rank_in_archive_order":10,"of":10,"metrics":{"Median Win Rate":"20.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-infantry-sequential","task":"SMAC+","dataset":"Def_Infantry_sequential","model":"DMIX","rank_in_archive_order":4,"of":11,"metrics":{"Median Win Rate":"100"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-infantry-sequential","task":"SMAC+","dataset":"Def_Infantry_sequential","model":"DIQL","rank_in_archive_order":8,"of":11,"metrics":{"Median Win Rate":"93.8"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-infantry-sequential","task":"SMAC+","dataset":"Def_Infantry_sequential","model":"DDN","rank_in_archive_order":9,"of":11,"metrics":{"Median Win Rate":"90.6"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-outnumbered-parallel","task":"SMAC+","dataset":"Def_Outnumbered_parallel","model":"DMIX","rank_in_archive_order":3,"of":10,"metrics":{"Median Win Rate":"5.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-outnumbered-parallel","task":"SMAC+","dataset":"Def_Outnumbered_parallel","model":"DDN","rank_in_archive_order":9,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-outnumbered-parallel","task":"SMAC+","dataset":"Def_Outnumbered_parallel","model":"DIQL","rank_in_archive_order":10,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-outnumbered-sequential","task":"SMAC+","dataset":"Def_Outnumbered_sequential","model":"DDN","rank_in_archive_order":9,"of":11,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-outnumbered-sequential","task":"SMAC+","dataset":"Def_Outnumbered_sequential","model":"DIQL","rank_in_archive_order":10,"of":11,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-def-outnumbered-sequential","task":"SMAC+","dataset":"Def_Outnumbered_sequential","model":"DMIX","rank_in_archive_order":11,"of":11,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-complicated-parallel","task":"SMAC+","dataset":"Off_Complicated_parallel","model":"DDN","rank_in_archive_order":8,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-complicated-parallel","task":"SMAC+","dataset":"Off_Complicated_parallel","model":"DIQL","rank_in_archive_order":9,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-complicated-parallel","task":"SMAC+","dataset":"Off_Complicated_parallel","model":"DMIX","rank_in_archive_order":10,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-distant-parallel","task":"SMAC+","dataset":"Off_Distant_parallel","model":"DDN","rank_in_archive_order":8,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-distant-parallel","task":"SMAC+","dataset":"Off_Distant_parallel","model":"DIQL","rank_in_archive_order":9,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-distant-parallel","task":"SMAC+","dataset":"Off_Distant_parallel","model":"DMIX","rank_in_archive_order":10,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-hard-parallel","task":"SMAC+","dataset":"Off_Hard_parallel","model":"DDN","rank_in_archive_order":8,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-hard-parallel","task":"SMAC+","dataset":"Off_Hard_parallel","model":"DIQL","rank_in_archive_order":9,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-hard-parallel","task":"SMAC+","dataset":"Off_Hard_parallel","model":"DMIX","rank_in_archive_order":10,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-near-parallel","task":"SMAC+","dataset":"Off_Near_parallel","model":"DDN","rank_in_archive_order":8,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-near-parallel","task":"SMAC+","dataset":"Off_Near_parallel","model":"DIQL","rank_in_archive_order":9,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-near-parallel","task":"SMAC+","dataset":"Off_Near_parallel","model":"DMIX","rank_in_archive_order":10,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-superhard-parallel","task":"SMAC+","dataset":"Off_Superhard_parallel","model":"DDN","rank_in_archive_order":7,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-superhard-parallel","task":"SMAC+","dataset":"Off_Superhard_parallel","model":"DIQL","rank_in_archive_order":8,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false},{"leaderboard":"/sota/smac-on-smac-off-superhard-parallel","task":"SMAC+","dataset":"Off_Superhard_parallel","model":"DMIX","rank_in_archive_order":9,"of":10,"metrics":{"Median Win Rate":"0.0"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2102.07936","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2102.07936"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/j3soon/dfac","reach":null}],"summary":{"ran":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"38a1d2c8a56f1d27","entry":"DMixer","repo":"j3soon/dfac","repo_kind":"official","path":"pymarl/src/modules/mixers/dmix.py","file_url":"https://github.com/j3soon/dfac/blob/HEAD/pymarl/src/modules/mixers/dmix.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"38a1d2c8a56f1d27"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}