{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2510-11824","title":"Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learning","arxiv_id":"2510.11824","date":"2025-10-13","proceeding":"NeurIPS","authors":["Simin Li","Zihao Mao","Hanxiao Li","Zonglei Jing","Zhuohang bian","Jun Guo","Li Wang","Zhuoran Han","Ruixiao Xu","Xin Yu","Chengdong Ma","Yuqing Ma","Bo An","Yaodong Yang","Weifeng Lv","Xianglong Liu"],"abstract":"In cooperative Multi-Agent Reinforcement Learning (MARL), it is a common practice to tune hyperparameters in ideal simulated environments to maximize cooperative performance. However, policies tuned for cooperation often fail to maintain robustness and resilience under real-world uncertainties. Building trustworthy MARL systems requires a deep understanding of robustness, which ensures stability under uncertainties, and resilience, the ability to recover from disruptions--a concept extensively studied in control systems but largely overlooked in MARL. In this paper, we present a large-scale empirical study comprising over 82,620 experiments to evaluate cooperation, robustness, and resilience in MARL across 4 real-world environments, 13 uncertainty types, and 15 hyperparameters. Our key findings are: (1) Under mild uncertainty, optimizing cooperation improves robustness and resilience, but this link weakens as perturbations intensify. Robustness and resilience also varies by algorithm and uncertainty type. (2) Robustness and resilience do not generalize across uncertainty modalities or agent scopes: policies robust to action noise for all agents may fail under observation noise on a single agent. (3) Hyperparameter tuning is critical for trustworthy MARL: surprisingly, standard practices like parameter sharing, GAE, and PopArt can hurt robustness, while early stopping, high critic learning rates, and Leaky ReLU consistently help. By optimizing hyperparameters only, we observe substantial improvement in cooperation, robustness and resilience across all MARL backbones, with the phenomenon also generalizing to robust MARL methods across these backbones. Code and results available at https://github.com/BUAA-TrustworthyMARL/adv_marl_benchmark .","url_abs":"https://arxiv.org/abs/2510.11824","url_pdf":"https://arxiv.org/pdf/2510.11824","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2510.11824","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2510.11824"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/BUAA-TrustworthyMARL/adv_marl_benchmark","reach":{"status":"ok"}},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/PKU-MARL/DexterousHands","reach":{"status":"ok","spdx":"Apache-2.0"}},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/cts198859/deeprl_network","reach":null}],"summary":{"ran":2,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"c99d4a67dd62d6b9","entry":"get_active_func","repo":"BUAA-TrustworthyMARL/adv_marl_benchmark","repo_kind":"found_in_text","path":"amb/utils/model_utils.py","file_url":"https://github.com/BUAA-TrustworthyMARL/adv_marl_benchmark/blob/HEAD/amb/utils/model_utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c99d4a67dd62d6b9"}},{"code_sha256_prefix":"343918f81a5b2114","entry":"get_init_method","repo":"BUAA-TrustworthyMARL/adv_marl_benchmark","repo_kind":"found_in_text","path":"amb/utils/model_utils.py","file_url":"https://github.com/BUAA-TrustworthyMARL/adv_marl_benchmark/blob/HEAD/amb/utils/model_utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"343918f81a5b2114"}},{"code_sha256_prefix":"0d38522e0dfa4e40","entry":"build_td_lambda_targets","repo":"BUAA-TrustworthyMARL/adv_marl_benchmark","repo_kind":"found_in_text","path":"amb/algorithms/coma.py","file_url":"https://github.com/BUAA-TrustworthyMARL/adv_marl_benchmark/blob/HEAD/amb/algorithms/coma.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0d38522e0dfa4e40"}},{"code_sha256_prefix":"45f77fabdb24e54a","entry":"init_device","repo":"BUAA-TrustworthyMARL/adv_marl_benchmark","repo_kind":"found_in_text","path":"amb/utils/model_utils.py","file_url":"https://github.com/BUAA-TrustworthyMARL/adv_marl_benchmark/blob/HEAD/amb/utils/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"45f77fabdb24e54a"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.MA","source":"arxiv_api"},"syntology_extracted_results":null}