{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-episodic-memory-utilization-of","title":"Efficient Episodic Memory Utilization of Cooperative Multi-Agent Reinforcement Learning","arxiv_id":"2403.01112","date":"2024-03-02","proceeding":null,"authors":["Hyungho Na","Yunkyeong Seo","Il-Chul Moon"],"abstract":"In cooperative multi-agent reinforcement learning (MARL), agents aim to achieve a common goal, such as defeating enemies or scoring a goal. Existing MARL algorithms are effective but still require significant learning time and often get trapped in local optima by complex tasks, subsequently failing to discover a goal-reaching policy. To address this, we introduce Efficient episodic Memory Utilization (EMU) for MARL, with two primary objectives: (a) accelerating reinforcement learning by leveraging semantically coherent memory from an episodic buffer and (b) selectively promoting desirable transitions to prevent local convergence. To achieve (a), EMU incorporates a trainable encoder/decoder structure alongside MARL, creating coherent memory embeddings that facilitate exploratory memory recall. To achieve (b), EMU introduces a novel reward structure called episodic incentive based on the desirability of states. This reward improves the TD target in Q-learning and acts as an additional incentive for desirable transitions. We provide theoretical support for the proposed incentive and demonstrate the effectiveness of EMU compared to conventional episodic control. The proposed method is evaluated in StarCraft II and Google Research Football, and empirical results indicate further performance improvement over state-of-the-art methods.","url_abs":"https://arxiv.org/abs/2403.01112v2","url_pdf":"https://arxiv.org/pdf/2403.01112v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-episodic-memory-utilization-of","repo_url":"https://github.com/hyunghona/emu","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"multi-agent-reinforcement-learning","task_name":"Multi-agent Reinforcement Learning"},{"task_slug":"q-learning","task_name":"Q-Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"starcraft","task_name":"Starcraft"},{"task_slug":"starcraft-ii","task_name":"Starcraft II"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"q-learning","method_name":"Q-Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.01112","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.01112"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/HyunghoNa/EMU","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hyunghona/emu","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8d357ae1b776477f","entry":"inverse_distance","repo":"hyunghona/emu","repo_kind":"official","path":"EMU_release_pymarl/src/components/episodic_memory_buffer.py","file_url":"https://github.com/hyunghona/emu/blob/HEAD/EMU_release_pymarl/src/components/episodic_memory_buffer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8d357ae1b776477f"}},{"code_sha256_prefix":"d482de9b36f4ffd3","entry":"Episodic_memory_buffer","repo":"hyunghona/emu","repo_kind":"official","path":"EMU_release_pymarl/src/components/episodic_memory_buffer.py","file_url":"https://github.com/hyunghona/emu/blob/HEAD/EMU_release_pymarl/src/components/episodic_memory_buffer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d482de9b36f4ffd3"}},{"code_sha256_prefix":"54fe626072101a59","entry":"LRU_KNN","repo":"hyunghona/emu","repo_kind":"official","path":"EMU_release_pymarl/src/components/episodic_memory_buffer.py","file_url":"https://github.com/hyunghona/emu/blob/HEAD/EMU_release_pymarl/src/components/episodic_memory_buffer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"54fe626072101a59"}},{"code_sha256_prefix":"a469457997a42ae9","entry":"LRU_KNN_STATE","repo":"hyunghona/emu","repo_kind":"official","path":"EMU_release_pymarl/src/components/episodic_memory_buffer.py","file_url":"https://github.com/hyunghona/emu/blob/HEAD/EMU_release_pymarl/src/components/episodic_memory_buffer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a469457997a42ae9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}