{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cooperate-or-collapse-emergence-of","title":"Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents","arxiv_id":"2404.16698","date":"2024-04-25","proceeding":null,"authors":["Giorgio Piatti","Zhijing Jin","Max Kleiman-Weiner","Bernhard Schölkopf","Mrinmaya Sachan","Rada Mihalcea"],"abstract":"As AI systems pervade human life, ensuring that large language models (LLMs) make safe decisions remains a significant challenge. We introduce the Governance of the Commons Simulation (GovSim), a generative simulation platform designed to study strategic interactions and cooperative decision-making in LLMs. In GovSim, a society of AI agents must collectively balance exploiting a common resource with sustaining it for future use. This environment enables the study of how ethical considerations, strategic planning, and negotiation skills impact cooperative outcomes. We develop an LLM-based agent architecture and test it with the leading open and closed LLMs. We find that all but the most powerful LLM agents fail to achieve a sustainable equilibrium in GovSim, with the highest survival rate below 54%. Ablations reveal that successful multi-agent communication between agents is critical for achieving cooperation in these cases. Furthermore, our analyses show that the failure to achieve sustainable cooperation in most LLMs stems from their inability to formulate and analyze hypotheses about the long-term effects of their actions on the equilibrium of the group. Finally, we show that agents that leverage \"Universalization\"-based reasoning, a theory of moral thinking, are able to achieve significantly better sustainability. Taken together, GovSim enables us to study the mechanisms that underlie sustainable self-government with specificity and scale. We open source the full suite of our research results, including the simulation environment, agent prompts, and a comprehensive web interface.","url_abs":"https://arxiv.org/abs/2404.16698v4","url_pdf":"https://arxiv.org/pdf/2404.16698v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cooperate-or-collapse-emergence-of","repo_url":"https://github.com/giorgiopiatti/govsim","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"cooperate-or-collapse-emergence-of","repo_url":"https://github.com/pedrocurvo/govsim","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"specificity","task_name":"Specificity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2404.16698","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.16698"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/giorgiopiatti/GovSim","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/giorgiopiatti/govsim","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pedrocurvo/govsim","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":7},"by_repo_kind":{"official":{"samples":7,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1a814f972f1c0f28","entry":"columns_non_relevant","repo":"giorgiopiatti/GovSim","repo_kind":"official","path":"simulation/analysis/preprocessing.py","file_url":"https://github.com/giorgiopiatti/GovSim/blob/HEAD/simulation/analysis/preprocessing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a814f972f1c0f28"}},{"code_sha256_prefix":"f059157a9b113e12","entry":"compute_survival_months_stats","repo":"giorgiopiatti/GovSim","repo_kind":"official","path":"simulation/analysis/plots.py","file_url":"https://github.com/giorgiopiatti/GovSim/blob/HEAD/simulation/analysis/plots.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f059157a9b113e12"}},{"code_sha256_prefix":"2b2428498b1de81a","entry":"flatten_yaml","repo":"giorgiopiatti/GovSim","repo_kind":"official","path":"simulation/analysis/preprocessing.py","file_url":"https://github.com/giorgiopiatti/GovSim/blob/HEAD/simulation/analysis/preprocessing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b2428498b1de81a"}},{"code_sha256_prefix":"65c7a8b0ffd45803","entry":"generate_colors","repo":"giorgiopiatti/GovSim","repo_kind":"official","path":"simulation/analysis/utils.py","file_url":"https://github.com/giorgiopiatti/GovSim/blob/HEAD/simulation/analysis/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65c7a8b0ffd45803"}},{"code_sha256_prefix":"0c689686bee2ab64","entry":"get_LLM_family","repo":"giorgiopiatti/GovSim","repo_kind":"official","path":"utils/charts.py","file_url":"https://github.com/giorgiopiatti/GovSim/blob/HEAD/utils/charts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0c689686bee2ab64"}},{"code_sha256_prefix":"e62384293fca0d42","entry":"get_model_size_version","repo":"giorgiopiatti/GovSim","repo_kind":"official","path":"utils/charts.py","file_url":"https://github.com/giorgiopiatti/GovSim/blob/HEAD/utils/charts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e62384293fca0d42"}},{"code_sha256_prefix":"e8de137875cb4f35","entry":"get_pretty_name_llm","repo":"giorgiopiatti/GovSim","repo_kind":"official","path":"utils/charts.py","file_url":"https://github.com/giorgiopiatti/GovSim/blob/HEAD/utils/charts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e8de137875cb4f35"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}