{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-monte-carlo-aixi-approximation","title":"A Monte Carlo AIXI Approximation","arxiv_id":"0909.0801","date":"2009-09-04","proceeding":null,"authors":["Joel Veness","Kee Siong Ng","Marcus Hutter","William Uther","David Silver"],"abstract":"This paper introduces a principled approach for the design of a scalable\ngeneral reinforcement learning agent. Our approach is based on a direct\napproximation of AIXI, a Bayesian optimality notion for general reinforcement\nlearning agents. Previously, it has been unclear whether the theory of AIXI\ncould motivate the design of practical algorithms. We answer this hitherto open\nquestion in the affirmative, by providing the first computationally feasible\napproximation to the AIXI agent. To develop our approximation, we introduce a\nnew Monte-Carlo Tree Search algorithm along with an agent-specific extension to\nthe Context Tree Weighting algorithm. Empirically, we present a set of\nencouraging results on a variety of stochastic and partially observable\ndomains. We conclude by proposing a number of directions for future research.","url_abs":"http://arxiv.org/abs/0909.0801v2","url_pdf":"http://arxiv.org/pdf/0909.0801v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-monte-carlo-aixi-approximation","repo_url":"https://github.com/gkassel/pyaixi","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"a-monte-carlo-aixi-approximation","repo_url":"https://github.com/sgkasselau/pyaixi","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"general-reinforcement-learning","task_name":"General Reinforcement Learning"},{"task_slug":"open-question","task_name":"Open-Ended Question Answering"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"monte-carlo-tree-search","method_name":"Monte-Carlo Tree Search"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}