{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-multi-agent-reinforcement-learning-model-of","title":"A multi-agent reinforcement learning model of common-pool resource appropriation","arxiv_id":"1707.06600","date":"2017-07-20","proceeding":"NeurIPS 2017 12","authors":["Julien Perolat","Joel Z. Leibo","Vinicius Zambaldi","Charles Beattie","Karl Tuyls","Thore Graepel"],"abstract":"Humanity faces numerous problems of common-pool resource appropriation. This\nclass of multi-agent social dilemma includes the problems of ensuring\nsustainable use of fresh water, common fisheries, grazing pastures, and\nirrigation systems. Abstract models of common-pool resource appropriation based\non non-cooperative game theory predict that self-interested agents will\ngenerally fail to find socially positive equilibria---a phenomenon called the\ntragedy of the commons. However, in reality, human societies are sometimes able\nto discover and implement stable cooperative solutions. Decades of behavioral\ngame theory research have sought to uncover aspects of human behavior that make\nthis possible. Most of that work was based on laboratory experiments where\nparticipants only make a single choice: how much to appropriate. Recognizing\nthe importance of spatial and temporal resource dynamics, a recent trend has\nbeen toward experiments in more complex real-time video game-like environments.\nHowever, standard methods of non-cooperative game theory can no longer be used\nto generate predictions for this case. Here we show that deep reinforcement\nlearning can be used instead. To that end, we study the emergent behavior of\ngroups of independently learning agents in a partially observed Markov game\nmodeling common-pool resource appropriation. Our experiments highlight the\nimportance of trial-and-error learning in common-pool resource appropriation\nand shed light on the relationship between exclusion, sustainability, and\ninequality.","url_abs":"http://arxiv.org/abs/1707.06600v2","url_pdf":"http://arxiv.org/pdf/1707.06600v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-multi-agent-reinforcement-learning-model-of","repo_url":"https://github.com/HumanCompatibleAI/multi-agent","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"a-multi-agent-reinforcement-learning-model-of","repo_url":"https://github.com/fabien-couthouis/xai-in-rl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"a-multi-agent-reinforcement-learning-model-of","repo_url":"https://github.com/tiagoCuervo/CommonsGame","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"a-multi-agent-reinforcement-learning-model-of","repo_url":"https://github.com/Wadaboa/cpr-appropriation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"multi-agent-reinforcement-learning","task_name":"Multi-agent Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1707.06600","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}