{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/madiff-offline-multi-agent-learning-with","title":"MADiff: Offline Multi-agent Learning with Diffusion Models","arxiv_id":"2305.17330","date":"2023-05-27","proceeding":null,"authors":["Zhengbang Zhu","Minghuan Liu","Liyuan Mao","Bingyi Kang","Minkai Xu","Yong Yu","Stefano Ermon","Weinan Zhang"],"abstract":"Offline reinforcement learning (RL) aims to learn policies from pre-existing datasets without further interactions, making it a challenging task. Q-learning algorithms struggle with extrapolation errors in offline settings, while supervised learning methods are constrained by model expressiveness. Recently, diffusion models (DMs) have shown promise in overcoming these limitations in single-agent learning, but their application in multi-agent scenarios remains unclear. Generating trajectories for each agent with independent DMs may impede coordination, while concatenating all agents' information can lead to low sample efficiency. Accordingly, we propose MADiff, which is realized with an attention-based diffusion model to model the complex coordination among behaviors of multiple agents. To our knowledge, MADiff is the first diffusion-based multi-agent learning framework, functioning as both a decentralized policy and a centralized controller. During decentralized executions, MADiff simultaneously performs teammate modeling, and the centralized controller can also be applied in multi-agent trajectory predictions. Our experiments demonstrate that MADiff outperforms baseline algorithms across various multi-agent learning tasks, highlighting its effectiveness in modeling complex multi-agent interactions. Our code is available at https://github.com/zbzhu99/madiff.","url_abs":"https://arxiv.org/abs/2305.17330v5","url_pdf":"https://arxiv.org/pdf/2305.17330v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"madiff-offline-multi-agent-learning-with","repo_url":"https://github.com/zbzhu99/madiff","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"offline-rl","task_name":"Offline RL"},{"task_slug":"q-learning","task_name":"Q-Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"trajectory-prediction","task_name":"Trajectory Prediction"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.17330","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.17330"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zbzhu99/madiff","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":6},"by_repo_kind":{"official":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2c6a85145878d112","entry":"apply_conditioning","repo":"zbzhu99/madiff","repo_kind":"official","path":"diffuser/models/helpers.py","file_url":"https://github.com/zbzhu99/madiff/blob/HEAD/diffuser/models/helpers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2c6a85145878d112"}},{"code_sha256_prefix":"1876c512557a08d5","entry":"atleast_nd","repo":"zbzhu99/madiff","repo_kind":"official","path":"diffuser/datasets/buffer.py","file_url":"https://github.com/zbzhu99/madiff/blob/HEAD/diffuser/datasets/buffer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1876c512557a08d5"}},{"code_sha256_prefix":"09c8479d9a5b3e06","entry":"extract","repo":"zbzhu99/madiff","repo_kind":"official","path":"diffuser/models/helpers.py","file_url":"https://github.com/zbzhu99/madiff/blob/HEAD/diffuser/models/helpers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"09c8479d9a5b3e06"}},{"code_sha256_prefix":"3faeb3b26f50eb5b","entry":"get_dataset","repo":"zbzhu99/madiff","repo_kind":"official","path":"diffuser/datasets/d4rl.py","file_url":"https://github.com/zbzhu99/madiff/blob/HEAD/diffuser/datasets/d4rl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3faeb3b26f50eb5b"}},{"code_sha256_prefix":"3b79a4ece470de64","entry":"load_environment","repo":"zbzhu99/madiff","repo_kind":"official","path":"diffuser/datasets/d4rl.py","file_url":"https://github.com/zbzhu99/madiff/blob/HEAD/diffuser/datasets/d4rl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b79a4ece470de64"}},{"code_sha256_prefix":"9da13e0388a251c5","entry":"pretend_multiagent","repo":"zbzhu99/madiff","repo_kind":"official","path":"diffuser/datasets/d4rl.py","file_url":"https://github.com/zbzhu99/madiff/blob/HEAD/diffuser/datasets/d4rl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9da13e0388a251c5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}