{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sampled-policy-gradient-for-learning-to-play","title":"Sampled Policy Gradient for Learning to Play the Game Agar.io","arxiv_id":"1809.05763","date":"2018-09-15","proceeding":null,"authors":["Anton Orell Wiehe","Nil Stolt Ansó","Madalina M. Drugan","Marco A. Wiering"],"abstract":"In this paper, a new offline actor-critic learning algorithm is introduced:\nSampled Policy Gradient (SPG). SPG samples in the action space to calculate an\napproximated policy gradient by using the critic to evaluate the samples. This\nsampling allows SPG to search the action-Q-value space more globally than\ndeterministic policy gradient (DPG), enabling it to theoretically avoid more\nlocal optima. SPG is compared to Q-learning and the actor-critic algorithms\nCACLA and DPG in a pellet collection task and a self play environment in the\ngame Agar.io. The online game Agar.io has become massively popular on the\ninternet due to intuitive game design and the ability to instantly compete\nagainst players around the world. From the point of view of artificial\nintelligence this game is also very intriguing: The game has a continuous input\nand action space and allows to have diverse agents with complex strategies\ncompete against each other. The experimental results show that Q-Learning and\nCACLA outperform a pre-programmed greedy bot in the pellet collection task, but\nall algorithms fail to outperform this bot in a fighting scenario. The SPG\nalgorithm is analyzed to have great extendability through offline exploration\nand it matches DPG in performance even in its basic form without extensive\nsampling.","url_abs":"http://arxiv.org/abs/1809.05763v1","url_pdf":"http://arxiv.org/pdf/1809.05763v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sampled-policy-gradient-for-learning-to-play","repo_url":"https://github.com/RUKip/MachineLearningProject","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"sampled-policy-gradient-for-learning-to-play","repo_url":"https://github.com/mrcabo/MachineLearningProject","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"game-design","task_name":"Game Design"},{"task_slug":"q-learning","task_name":"Q-Learning"}],"methods":[{"method_slug":"dpg","method_name":"DPG"},{"method_slug":"q-learning","method_name":"Q-Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1809.05763","atlas_url":"https://app.syntology.ai/?focus=1809.05763","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}