{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/pmic-improving-multi-agent-reinforcement-1","title":"PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaboration","arxiv_id":"2203.08553","date":"2022-03-16","proceeding":null,"authors":["Pengyi Li","Hongyao Tang","Tianpei Yang","Xiaotian Hao","Tong Sang","Yan Zheng","Jianye Hao","Matthew E. Taylor","Wenyuan Tao","Zhen Wang","Fazl Barez"],"abstract":"Learning to collaborate is critical in Multi-Agent Reinforcement Learning (MARL). Previous works promote collaboration by maximizing the correlation of agents' behaviors, which is typically characterized by Mutual Information (MI) in different forms. However, we reveal sub-optimal collaborative behaviors also emerge with strong correlations, and simply maximizing the MI can, surprisingly, hinder the learning towards better collaboration. To address this issue, we propose a novel MARL framework, called Progressive Mutual Information Collaboration (PMIC), for more effective MI-driven collaboration. PMIC uses a new collaboration criterion measured by the MI between global states and joint actions. Based on this criterion, the key idea of PMIC is maximizing the MI associated with superior collaborative behaviors and minimizing the MI associated with inferior ones. The two MI objectives play complementary roles by facilitating better collaborations while avoiding falling into sub-optimal ones. Experiments on a wide range of MARL benchmarks show the superior performance of PMIC compared with other algorithms.","url_abs":"https://arxiv.org/abs/2203.08553v4","url_pdf":"https://arxiv.org/pdf/2203.08553v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"pmic-improving-multi-agent-reinforcement-1","repo_url":"https://github.com/yeshenpy/pmic","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"multi-agent-reinforcement-learning","task_name":"Multi-agent Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.08553","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.08553"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/yeshenpy/PMIC","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yeshenpy/pmic","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"ran_draft_wrong":1,"ran_fixture":2,"unverified":1},"by_repo_kind":{"official":{"samples":9,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"02d8b297accf97de","entry":"Actor","repo":"yeshenpy/pmic","repo_kind":"official","path":"algorithms/mpe_new_maxminMADDPG.py","file_url":"https://github.com/yeshenpy/pmic/blob/HEAD/algorithms/mpe_new_maxminMADDPG.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"02d8b297accf97de"}},{"code_sha256_prefix":"93f26c6f16a8c0ae","entry":"CLUB","repo":"yeshenpy/pmic","repo_kind":"official","path":"algorithms/mpe_new_maxminMADDPG.py","file_url":"https://github.com/yeshenpy/pmic/blob/HEAD/algorithms/mpe_new_maxminMADDPG.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"93f26c6f16a8c0ae"}},{"code_sha256_prefix":"0b5742fd1596a2a9","entry":"Critic","repo":"yeshenpy/pmic","repo_kind":"official","path":"algorithms/mpe_new_maxminMADDPG.py","file_url":"https://github.com/yeshenpy/pmic/blob/HEAD/algorithms/mpe_new_maxminMADDPG.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b5742fd1596a2a9"}},{"code_sha256_prefix":"fe832e6e436cb7d8","entry":"NEW_MINE","repo":"yeshenpy/pmic","repo_kind":"official","path":"algorithms/mpe_new_maxminMADDPG.py","file_url":"https://github.com/yeshenpy/pmic/blob/HEAD/algorithms/mpe_new_maxminMADDPG.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe832e6e436cb7d8"}},{"code_sha256_prefix":"5a6b141fd404c7c4","entry":"OUNoise","repo":"yeshenpy/pmic","repo_kind":"official","path":"algorithms/mpe_new_maxminMADDPG.py","file_url":"https://github.com/yeshenpy/pmic/blob/HEAD/algorithms/mpe_new_maxminMADDPG.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5a6b141fd404c7c4"}},{"code_sha256_prefix":"dc127827c3f6691e","entry":"fenchel_dual_loss","repo":"yeshenpy/pmic","repo_kind":"official","path":"algorithms/mpe_new_maxminMADDPG.py","file_url":"https://github.com/yeshenpy/pmic/blob/HEAD/algorithms/mpe_new_maxminMADDPG.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dc127827c3f6691e"}},{"code_sha256_prefix":"640aa7bb79c50862","entry":"get_negative_expectation","repo":"yeshenpy/pmic","repo_kind":"official","path":"algorithms/mpe_new_maxminMADDPG.py","file_url":"https://github.com/yeshenpy/pmic/blob/HEAD/algorithms/mpe_new_maxminMADDPG.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"640aa7bb79c50862"}},{"code_sha256_prefix":"0edef8661a209cf5","entry":"get_positive_expectation","repo":"yeshenpy/pmic","repo_kind":"official","path":"algorithms/mpe_new_maxminMADDPG.py","file_url":"https://github.com/yeshenpy/pmic/blob/HEAD/algorithms/mpe_new_maxminMADDPG.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0edef8661a209cf5"}},{"code_sha256_prefix":"96cedade50014de1","entry":"MA_T_DDPG","repo":"yeshenpy/pmic","repo_kind":"official","path":"algorithms/mpe_new_maxminMADDPG.py","file_url":"https://github.com/yeshenpy/pmic/blob/HEAD/algorithms/mpe_new_maxminMADDPG.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"96cedade50014de1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}