{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/iterated-reasoning-with-mutual-information-in-1","title":"Iterated Reasoning with Mutual Information in Cooperative and Byzantine Decentralized Teaming","arxiv_id":"2201.08484","date":"2022-01-20","proceeding":"ICLR 2022 4","authors":["Sachin Konan","Esmaeil Seraj","Matthew Gombolay"],"abstract":"Information sharing is key in building team cognition and enables coordination and cooperation. High-performing human teams also benefit from acting strategically with hierarchical levels of iterated communication and rationalizability, meaning a human agent can reason about the actions of their teammates in their decision-making. Yet, the majority of prior work in Multi-Agent Reinforcement Learning (MARL) does not support iterated rationalizability and only encourage inter-agent communication, resulting in a suboptimal equilibrium cooperation strategy. In this work, we show that reformulating an agent's policy to be conditional on the policies of its neighboring teammates inherently maximizes Mutual Information (MI) lower-bound when optimizing under Policy Gradient (PG). Building on the idea of decision-making under bounded rationality and cognitive hierarchy theory, we show that our modified PG approach not only maximizes local agent rewards but also implicitly reasons about MI between agents without the need for any explicit ad-hoc regularization terms. Our approach, InfoPG, outperforms baselines in learning emergent collaborative behaviors and sets the state-of-the-art in decentralized cooperative MARL tasks. Our experiments validate the utility of InfoPG by achieving higher sample efficiency and significantly larger cumulative reward in several complex cooperative multi-agent domains.","url_abs":"https://arxiv.org/abs/2201.08484v4","url_pdf":"https://arxiv.org/pdf/2201.08484v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"iterated-reasoning-with-mutual-information-in-1","repo_url":"https://github.com/core-robotics-lab/infopg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"multi-agent-reinforcement-learning","task_name":"Multi-agent Reinforcement Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2201.08484","atlas_url":"https://app.syntology.ai/?focus=2201.08484","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2201.08484"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/CORE-Robotics-Lab/InfoPG","reach":{"status":"ok","spdx":"GPL-3.0"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/deepmind/pysc2","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/core-robotics-lab/infopg","reach":{"status":"ok","spdx":"GPL-3.0"}}],"summary":{"ran":7,"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1},"found_in_text":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"593ef84a0fec9ab2","entry":"MOA_PolicyHelper","repo":"core-robotics-lab/infopg","repo_kind":"official","path":"pistonball/policy_piston.py","file_url":"https://github.com/core-robotics-lab/infopg/blob/HEAD/pistonball/policy_piston.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"593ef84a0fec9ab2"}},{"code_sha256_prefix":"1314ce1206a217e8","entry":"memoize","repo":"deepmind/pysc2","repo_kind":"found_in_text","path":"pysc2/lib/memoize.py","file_url":"https://github.com/deepmind/pysc2/blob/HEAD/pysc2/lib/memoize.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"1314ce1206a217e8"}},{"code_sha256_prefix":"795b48aa13444f75","entry":"numpy_to_python","repo":"deepmind/pysc2","repo_kind":"found_in_text","path":"pysc2/lib/actions.py","file_url":"https://github.com/deepmind/pysc2/blob/HEAD/pysc2/lib/actions.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"795b48aa13444f75"}},{"code_sha256_prefix":"1902983401727afd","entry":"piece_wise_linear","repo":"deepmind/pysc2","repo_kind":"found_in_text","path":"pysc2/lib/colors.py","file_url":"https://github.com/deepmind/pysc2/blob/HEAD/pysc2/lib/colors.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"1902983401727afd"}},{"code_sha256_prefix":"094587ca8ce60a1a","entry":"shuffled_hue","repo":"deepmind/pysc2","repo_kind":"found_in_text","path":"pysc2/lib/colors.py","file_url":"https://github.com/deepmind/pysc2/blob/HEAD/pysc2/lib/colors.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"094587ca8ce60a1a"}},{"code_sha256_prefix":"ee4a38c9d2b4ba59","entry":"smooth_hue_palette","repo":"deepmind/pysc2","repo_kind":"found_in_text","path":"pysc2/lib/colors.py","file_url":"https://github.com/deepmind/pysc2/blob/HEAD/pysc2/lib/colors.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"ee4a38c9d2b4ba59"}},{"code_sha256_prefix":"b9be4b0c3dee19d6","entry":"summarize_array_diffs","repo":"deepmind/pysc2","repo_kind":"found_in_text","path":"pysc2/lib/np_util.py","file_url":"https://github.com/deepmind/pysc2/blob/HEAD/pysc2/lib/np_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"b9be4b0c3dee19d6"}},{"code_sha256_prefix":"93e90c9405886816","entry":"spatial","repo":"deepmind/pysc2","repo_kind":"found_in_text","path":"pysc2/lib/actions.py","file_url":"https://github.com/deepmind/pysc2/blob/HEAD/pysc2/lib/actions.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":false,"mcp_get_code":{"code_sha256":"93e90c9405886816"}},{"code_sha256_prefix":"d04972da01192ed6","entry":"train_adv_infopg_5","repo":"core-robotics-lab/infopg","repo_kind":"official","path":"pistonball/test_piston_ball.py","file_url":"https://github.com/core-robotics-lab/infopg/blob/HEAD/pistonball/test_piston_ball.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"d04972da01192ed6"}},{"code_sha256_prefix":"8673d75351fb3782","entry":"train_infopg_5","repo":"core-robotics-lab/infopg","repo_kind":"official","path":"pistonball/test_piston_ball.py","file_url":"https://github.com/core-robotics-lab/infopg/blob/HEAD/pistonball/test_piston_ball.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"8673d75351fb3782"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}