{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-multi-agent-reinforcement-learning-for","title":"FACMAC: Factored Multi-Agent Centralised Policy Gradients","arxiv_id":"2003.06709","date":"2020-03-14","proceeding":"NeurIPS 2021 12","authors":["Bei Peng","Tabish Rashid","Christian A. Schroeder de Witt","Pierre-Alexandre Kamienny","Philip H. S. Torr","Wendelin Böhmer","Shimon Whiteson"],"abstract":"We propose FACtored Multi-Agent Centralised policy gradients (FACMAC), a new method for cooperative multi-agent reinforcement learning in both discrete and continuous action spaces. Like MADDPG, a popular multi-agent actor-critic method, our approach uses deep deterministic policy gradients to learn policies. However, FACMAC learns a centralised but factored critic, which combines per-agent utilities into the joint action-value function via a non-linear monotonic function, as in QMIX, a popular multi-agent Q-learning algorithm. However, unlike QMIX, there are no inherent constraints on factoring the critic. We thus also employ a nonmonotonic factorisation and empirically demonstrate that its increased representational capacity allows it to solve some tasks that cannot be solved with monolithic, or monotonically factored critics. In addition, FACMAC uses a centralised policy gradient estimator that optimises over the entire joint action space, rather than optimising over each agent's action space separately as in MADDPG. This allows for more coordinated policy changes and fully reaps the benefits of a centralised critic. We evaluate FACMAC on variants of the multi-agent particle environments, a novel multi-agent MuJoCo benchmark, and a challenging set of StarCraft II micromanagement tasks. Empirical results demonstrate FACMAC's superior performance over MADDPG and other baselines on all three domains.","url_abs":"https://arxiv.org/abs/2003.06709v5","url_pdf":"https://arxiv.org/pdf/2003.06709v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-multi-agent-reinforcement-learning-for","repo_url":"https://github.com/schroederdewitt/multiagent_mujoco","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"deep-multi-agent-reinforcement-learning-for","repo_url":"https://github.com/oxwhirl/comix","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"deep-multi-agent-reinforcement-learning-for","repo_url":"https://github.com/oxwhirl/facmac","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"mujoco","task_name":"MuJoCo"},{"task_slug":"multi-agent-reinforcement-learning","task_name":"Multi-agent Reinforcement Learning"},{"task_slug":"q-learning","task_name":"Q-Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"smac","task_name":"SMAC"},{"task_slug":"starcraft","task_name":"Starcraft"},{"task_slug":"starcraft-ii","task_name":"Starcraft II"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"experience-replay","method_name":"Experience Replay"},{"method_slug":"maddpg","method_name":"MADDPG"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2003.06709","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2003.06709"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/schroederdewitt/multiagent_mujoco","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/oxwhirl/comix","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/oxwhirl/facmac","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9d0e7c23576a092f","entry":"convert","repo":"schroederdewitt/multiagent_mujoco","repo_kind":"official","path":"multiagent_mujoco/multiagentenv.py","file_url":"https://github.com/schroederdewitt/multiagent_mujoco/blob/HEAD/multiagent_mujoco/multiagentenv.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9d0e7c23576a092f"}},{"code_sha256_prefix":"7b68f30bd920cce1","entry":"build_obs","repo":"schroederdewitt/multiagent_mujoco","repo_kind":"official","path":"multiagent_mujoco/obsk.py","file_url":"https://github.com/schroederdewitt/multiagent_mujoco/blob/HEAD/multiagent_mujoco/obsk.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7b68f30bd920cce1"}},{"code_sha256_prefix":"b740cb6f99054211","entry":"get_joints_at_kdist","repo":"schroederdewitt/multiagent_mujoco","repo_kind":"official","path":"multiagent_mujoco/obsk.py","file_url":"https://github.com/schroederdewitt/multiagent_mujoco/blob/HEAD/multiagent_mujoco/obsk.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b740cb6f99054211"}},{"code_sha256_prefix":"f61ec31ccd82ad82","entry":"get_parts_and_edges","repo":"schroederdewitt/multiagent_mujoco","repo_kind":"official","path":"multiagent_mujoco/obsk.py","file_url":"https://github.com/schroederdewitt/multiagent_mujoco/blob/HEAD/multiagent_mujoco/obsk.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f61ec31ccd82ad82"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}