{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-cooperate-with-humans-using","title":"Learning to Cooperate with Humans using Generative Agents","arxiv_id":"2411.13934","date":"2024-11-21","proceeding":null,"authors":["Yancheng Liang","Daphne Chen","Abhishek Gupta","Simon S. Du","Natasha Jaques"],"abstract":"Training agents that can coordinate zero-shot with humans is a key mission in multi-agent reinforcement learning (MARL). Current algorithms focus on training simulated human partner policies which are then used to train a Cooperator agent. The simulated human is produced either through behavior cloning over a dataset of human cooperation behavior, or by using MARL to create a population of simulated agents. However, these approaches often struggle to produce a Cooperator that can coordinate well with real humans, since the simulated humans fail to cover the diverse strategies and styles employed by people in the real world. We show \\emph{learning a generative model of human partners} can effectively address this issue. Our model learns a latent variable representation of the human that can be regarded as encoding the human's unique strategy, intention, experience, or style. This generative model can be flexibly trained from any (human or neural policy) agent interaction data. By sampling from the latent space, we can use the generative model to produce different partners to train Cooperator agents. We evaluate our method -- \\textbf{G}enerative \\textbf{A}gent \\textbf{M}odeling for \\textbf{M}ulti-agent \\textbf{A}daptation (GAMMA) -- on Overcooked, a challenging cooperative cooking game that has become a standard benchmark for zero-shot coordination. We conduct an evaluation with real human teammates, and the results show that GAMMA consistently improves performance, whether the generative model is trained on simulated populations or human datasets. Further, we propose a method for posterior sampling from the generative model that is biased towards the human data, enabling us to efficiently improve performance with only a small amount of expensive human interaction data.","url_abs":"https://arxiv.org/abs/2411.13934v1","url_pdf":"https://arxiv.org/pdf/2411.13934v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-cooperate-with-humans-using","repo_url":"https://github.com/lych1233/gamma-human-ai-collaboration","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"multi-agent-reinforcement-learning","task_name":"Multi-agent Reinforcement Learning"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2411.13934","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.13934"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lych1233/gamma-human-ai-collaboration","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"dae8ee0801414493","entry":"ScaledDotProductAttention","repo":"lych1233/gamma-human-ai-collaboration","repo_kind":"official","path":"mapbt/algorithms/utils/attention.py","file_url":"https://github.com/lych1233/gamma-human-ai-collaboration/blob/HEAD/mapbt/algorithms/utils/attention.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dae8ee0801414493"}},{"code_sha256_prefix":"57efc99646843520","entry":"add_path_prefix","repo":"lych1233/gamma-human-ai-collaboration","repo_kind":"official","path":"mapbt/algorithms/population/policy_pool.py","file_url":"https://github.com/lych1233/gamma-human-ai-collaboration/blob/HEAD/mapbt/algorithms/population/policy_pool.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57efc99646843520"}},{"code_sha256_prefix":"ad61f1e0e2ae04ed","entry":"split_obs","repo":"lych1233/gamma-human-ai-collaboration","repo_kind":"official","path":"mapbt/algorithms/utils/attention.py","file_url":"https://github.com/lych1233/gamma-human-ai-collaboration/blob/HEAD/mapbt/algorithms/utils/attention.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad61f1e0e2ae04ed"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}