{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/implicit-communication-as-minimum-entropy","title":"Communicating via Markov Decision Processes","arxiv_id":"2107.08295","date":"2021-07-17","proceeding":null,"authors":["Samuel Sokota","Christian Schroeder de Witt","Maximilian Igl","Luisa Zintgraf","Philip Torr","Martin Strohmeier","J. Zico Kolter","Shimon Whiteson","Jakob Foerster"],"abstract":"We consider the problem of communicating exogenous information by means of Markov decision process trajectories. This setting, which we call a Markov coding game (MCG), generalizes both source coding and a large class of referential games. MCGs also isolate a problem that is important in decentralized control settings in which cheap-talk is not available -- namely, they require balancing communication with the associated cost of communicating. We contribute a theoretically grounded approach to MCGs based on maximum entropy reinforcement learning and minimum entropy coupling that we call MEME. Due to recent breakthroughs in approximation algorithms for minimum entropy coupling, MEME is not merely a theoretical algorithm, but can be applied to practical settings. Empirically, we show both that MEME is able to outperform a strong baseline on small MCGs and that MEME is able to achieve strong performance on extremely large MCGs. To the latter point, we demonstrate that MEME is able to losslessly communicate binary images via trajectories of Cartpole and Pong, while simultaneously achieving the maximal or near maximal expected returns, and that it is even capable of performing well in the presence of actuator noise.","url_abs":"https://arxiv.org/abs/2107.08295v2","url_pdf":"https://arxiv.org/pdf/2107.08295v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"implicit-communication-as-minimum-entropy","repo_url":"https://github.com/schroederdewitt/meme","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"multi-agent-reinforcement-learning","task_name":"Multi-agent Reinforcement Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2107.08295","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2107.08295"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/schroederdewitt/meme","reach":null}],"summary":{"ran_fixture":2,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1d1b5ec53efcb7b7","entry":"Lemma3","repo":"schroederdewitt/meme","repo_kind":"official","path":"src/minimum_entropy_utils.py","file_url":"https://github.com/schroederdewitt/meme/blob/HEAD/src/minimum_entropy_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1d1b5ec53efcb7b7"}},{"code_sha256_prefix":"1171ee49436edbbe","entry":"greatest_lower_bound","repo":"schroederdewitt/meme","repo_kind":"official","path":"src/minimum_entropy_utils.py","file_url":"https://github.com/schroederdewitt/meme/blob/HEAD/src/minimum_entropy_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1171ee49436edbbe"}},{"code_sha256_prefix":"5386a45e20e1af97","entry":"load_image_to_token","repo":"schroederdewitt/meme","repo_kind":"official","path":"src/minimum_entropy.py","file_url":"https://github.com/schroederdewitt/meme/blob/HEAD/src/minimum_entropy.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5386a45e20e1af97"}},{"code_sha256_prefix":"88efedce36b9b019","entry":"minimum_entropy_coupling","repo":"schroederdewitt/meme","repo_kind":"official","path":"src/minimum_entropy_utils.py","file_url":"https://github.com/schroederdewitt/meme/blob/HEAD/src/minimum_entropy_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"88efedce36b9b019"}},{"code_sha256_prefix":"e50631fd4dc5963e","entry":"get_free_device_memory","repo":"schroederdewitt/meme","repo_kind":"official","path":"src/train_ff.py","file_url":"https://github.com/schroederdewitt/meme/blob/HEAD/src/train_ff.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e50631fd4dc5963e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}