{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/aixijs-a-software-demo-for-general","title":"AIXIjs: A Software Demo for General Reinforcement Learning","arxiv_id":"1705.07615","date":"2017-05-22","proceeding":null,"authors":["John Aslanides"],"abstract":"Reinforcement learning is a general and powerful framework with which to\nstudy and implement artificial intelligence. Recent advances in deep learning\nhave enabled RL algorithms to achieve impressive performance in restricted\ndomains such as playing Atari video games (Mnih et al., 2015) and, recently,\nthe board game Go (Silver et al., 2016). However, we are still far from\nconstructing a generally intelligent agent. Many of the obstacles and open\nquestions are conceptual: What does it mean to be intelligent? How does one\nexplore and learn optimally in general, unknown environments? What, in fact,\ndoes it mean to be optimal in the general sense? The universal Bayesian agent\nAIXI (Hutter, 2005) is a model of a maximally intelligent agent, and plays a\ncentral role in the sub-field of general reinforcement learning (GRL).\nRecently, AIXI has been shown to be flawed in important ways; it doesn't\nexplore enough to be asymptotically optimal (Orseau, 2010), and it can perform\npoorly with certain priors (Leike and Hutter, 2015). Several variants of AIXI\nhave been proposed to attempt to address these shortfalls: among them are\nentropy-seeking agents (Orseau, 2011), knowledge-seeking agents (Orseau et al.,\n2013), Bayes with bursts of exploration (Lattimore, 2013), MDL agents (Leike,\n2016a), Thompson sampling (Leike et al., 2016), and optimism (Sunehag and\nHutter, 2015). We present AIXIjs, a JavaScript implementation of these GRL\nagents. This implementation is accompanied by a framework for running\nexperiments against various environments, similar to OpenAI Gym (Brockman et\nal., 2016), and a suite of interactive demos that explore different properties\nof the agents, similar to REINFORCEjs (Karpathy, 2015). We use AIXIjs to\npresent numerous experiments illustrating fundamental properties of, and\ndifferences between, these agents.","url_abs":"http://arxiv.org/abs/1705.07615v1","url_pdf":"http://arxiv.org/pdf/1705.07615v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"aixijs-a-software-demo-for-general","repo_url":"https://github.com/aslanides/aixijs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"general-reinforcement-learning","task_name":"General Reinforcement Learning"},{"task_slug":"openai-gym","task_name":"OpenAI Gym"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"thompson-sampling","task_name":"Thompson Sampling"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"mdl","method_name":"MDL"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}