{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/offline-multi-agent-reinforcement-learning-1","title":"Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularization","arxiv_id":"2307.11620","date":"2023-07-21","proceeding":"NeurIPS 2023 11","authors":["Xiangsen Wang","Haoran Xu","Yinan Zheng","Xianyuan Zhan"],"abstract":"Offline reinforcement learning (RL) has received considerable attention in recent years due to its attractive capability of learning policies from offline datasets without environmental interactions. Despite some success in the single-agent setting, offline multi-agent RL (MARL) remains to be a challenge. The large joint state-action space and the coupled multi-agent behaviors pose extra complexities for offline policy optimization. Most existing offline MARL studies simply apply offline data-related regularizations on individual agents, without fully considering the multi-agent system at the global level. In this work, we present OMIGA, a new offline m ulti-agent RL algorithm with implicit global-to-local v alue regularization. OMIGA provides a principled framework to convert global-level value regularization into equivalent implicit local value regularizations and simultaneously enables in-sample learning, thus elegantly bridging multi-agent value decomposition and policy learning with offline regularizations. Based on comprehensive experiments on the offline multi-agent MuJoCo and StarCraft II micro-management tasks, we show that OMIGA achieves superior performance over the state-of-the-art offline MARL methods in almost all tasks.","url_abs":"https://arxiv.org/abs/2307.11620v2","url_pdf":"https://arxiv.org/pdf/2307.11620v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"offline-multi-agent-reinforcement-learning-1","repo_url":"https://github.com/zhengyinan-air/omiga","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"management","task_name":"Management"},{"task_slug":"mujoco","task_name":"MuJoCo"},{"task_slug":"multi-agent-reinforcement-learning","task_name":"Multi-agent Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"starcraft","task_name":"Starcraft"},{"task_slug":"starcraft-ii","task_name":"Starcraft II"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2307.11620","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.11620"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ZhengYinan-AIR/OMIGA","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhengyinan-air/omiga","reach":null}],"summary":{"ran":4,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"5eb07edf75209baa","entry":"Actor","repo":"zhengyinan-air/omiga","repo_kind":"official","path":"algos/OMIGA.py","file_url":"https://github.com/zhengyinan-air/omiga/blob/HEAD/algos/OMIGA.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5eb07edf75209baa"}},{"code_sha256_prefix":"065e9d8ef41ce92a","entry":"MixNet","repo":"zhengyinan-air/omiga","repo_kind":"official","path":"algos/OMIGA.py","file_url":"https://github.com/zhengyinan-air/omiga/blob/HEAD/algos/OMIGA.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"065e9d8ef41ce92a"}},{"code_sha256_prefix":"57f6a7164721cffa","entry":"Q_critic","repo":"zhengyinan-air/omiga","repo_kind":"official","path":"algos/OMIGA.py","file_url":"https://github.com/zhengyinan-air/omiga/blob/HEAD/algos/OMIGA.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"57f6a7164721cffa"}},{"code_sha256_prefix":"92ad543afdf6f2c6","entry":"V_critic","repo":"zhengyinan-air/omiga","repo_kind":"official","path":"algos/OMIGA.py","file_url":"https://github.com/zhengyinan-air/omiga/blob/HEAD/algos/OMIGA.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"92ad543afdf6f2c6"}},{"code_sha256_prefix":"771eca3c59698c8d","entry":"OMIGA","repo":"zhengyinan-air/omiga","repo_kind":"official","path":"algos/OMIGA.py","file_url":"https://github.com/zhengyinan-air/omiga/blob/HEAD/algos/OMIGA.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"771eca3c59698c8d"}},{"code_sha256_prefix":"3b81875ee24223a6","entry":"boolean","repo":"ZhengYinan-AIR/OMIGA","repo_kind":"official","path":"configs/config.py","file_url":"https://github.com/ZhengYinan-AIR/OMIGA/blob/HEAD/configs/config.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3b81875ee24223a6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}