{"url":"/method/monte-carlo-tree-search","slug":"monte-carlo-tree-search","name":"Monte-Carlo Tree Search","full_name":"Monte-Carlo Tree Search","full_name_withheld":false,"description_markdown":"**Monte-Carlo Tree Search** is a planning algorithm that accumulates value estimates obtained from Monte Carlo simulations in order to successively direct simulations towards more highly-rewarded trajectories. We execute MCTS after encountering each new state to select an agent's action for that state: it is executed again to select the action for the next state. Each execution is an iterative process that simulates many trajectories starting from the current state to the terminal state. The core idea is to successively focus multiple simulations starting at the current state by extending the initial portions of trajectories that have received high evaluations from earlier simulations.\r\n\r\nSource: Sutton and Barto, Reinforcement Learning (2nd Edition)\r\n\r\nImage Credit: [Chaslot et al](https://www.aaai.org/Papers/AIIDE/2008/AIIDE08-036.pdf)","description_state":"present","introduced_year":2006,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Reinforcement Learning","area_id":"reinforcement-learning","collection":"Heuristic Search Algorithms","url":"/methods/category/heuristic-search-algorithms","pwc_aliases":[]}],"n_papers_tagged":166,"archive_num_papers":166,"papers_newest_first":[{"paper":null,"title":"VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agents","date":"2025-06-17","arxiv_id":"2506.21582","n_code_links":0,"syntology":null},{"paper":null,"title":"Calibrated Value-Aware Model Learning with Stochastic Environment Models","date":"2025-05-28","arxiv_id":"2505.22772","n_code_links":0,"syntology":null},{"paper":null,"title":"Solving General-Utility Markov Decision Processes in the Single-Trial Regime with Online Planning","date":"2025-05-21","arxiv_id":"2505.15782","n_code_links":0,"syntology":null},{"paper":null,"title":"Adaptive Stress Testing Black-Box LLM Planners","date":"2025-05-08","arxiv_id":"2505.05665","n_code_links":0,"syntology":null},{"paper":"/paper/adaptive-branch-and-bound-tree-exploration","title":"Adaptive Branch-and-Bound Tree Exploration for Neural Network Verification","date":"2025-05-02","arxiv_id":"2505.00963","n_code_links":1,"syntology":null},{"paper":"/paper/trans-zero-self-play-incentivizes-large","title":"Trans-Zero: Self-Play Incentivizes Large Language Models for Multilingual Translation Without Parallel Data","date":"2025-04-20","arxiv_id":"2504.14669","n_code_links":1,"syntology":null},{"paper":"/paper/neural-guided-equation-discovery","title":"Neural-Guided Equation Discovery","date":"2025-03-21","arxiv_id":"2503.16953","n_code_links":1,"syntology":null},{"paper":"/paper/a-neural-symbolic-model-for-space-physics","title":"A Neural Symbolic Model for Space Physics","date":"2025-03-11","arxiv_id":"2503.07994","n_code_links":1,"syntology":null},{"paper":"/paper/optionzero-planning-with-learned-options","title":"OptionZero: Planning with Learned Options","date":"2025-02-23","arxiv_id":"2502.16634","n_code_links":1,"syntology":{"ran":7,"of":8,"unverified":1,"pointer_only":8}},{"paper":null,"title":"Boost, Disentangle, and Customize: A Robust System2-to-System1 Pipeline for Code Generation","date":"2025-02-18","arxiv_id":"2502.12492","n_code_links":0,"syntology":null},{"paper":null,"title":"Reinforcement Learning in Strategy-Based and Atari Games: A Review of Google DeepMinds Innovations","date":"2025-02-14","arxiv_id":"2502.10303","n_code_links":0,"syntology":null},{"paper":null,"title":"Evaluating World Models with LLM for Decision Making","date":"2024-11-13","arxiv_id":"2411.08794","n_code_links":0,"syntology":null},{"paper":null,"title":"Evaluating Robustness of Reinforcement Learning Algorithms for Autonomous Shipping","date":"2024-11-07","arxiv_id":"2411.04915","n_code_links":0,"syntology":null},{"paper":null,"title":"Interpreting the Learned Model in MuZero Planning","date":"2024-11-07","arxiv_id":"2411.04580","n_code_links":0,"syntology":null},{"paper":"/paper/human-aligned-chess-with-a-bit-of-search","title":"Human-aligned Chess with a Bit of Search","date":"2024-10-04","arxiv_id":"2410.03893","n_code_links":1,"syntology":null},{"paper":null,"title":"Zero-Shot Multi-Hop Question Answering via Monte-Carlo Tree Search with Large Language Models","date":"2024-09-28","arxiv_id":"2409.19382","n_code_links":0,"syntology":null},{"paper":null,"title":"An Efficient and Generalizable Symbolic Regression Method for Time Series Analysis","date":"2024-09-06","arxiv_id":"2409.03986","n_code_links":0,"syntology":null},{"paper":"/paper/deepseek-prover-v1-5-harnessing-proof","title":"DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search","date":"2024-08-15","arxiv_id":"2408.08152","n_code_links":2,"syntology":{"ran":4,"of":10,"unverified":6,"pointer_only":0}},{"paper":null,"title":"Combining AI Control Systems and Human Decision Support via Robustness and Criticality","date":"2024-07-03","arxiv_id":"2407.03210","n_code_links":0,"syntology":null},{"paper":null,"title":"Enhancements for Real-Time Monte-Carlo Tree Search in General Video Game Playing","date":"2024-07-03","arxiv_id":"2407.03049","n_code_links":0,"syntology":null},{"paper":null,"title":"Towards a Characterisation of Monte-Carlo Tree Search Performance in Different Games","date":"2024-06-13","arxiv_id":"2406.09242","n_code_links":0,"syntology":null},{"paper":"/paper/efficient-monte-carlo-tree-search-via-on-the","title":"Efficient Monte Carlo Tree Search via On-the-Fly State-Conditioned Action Abstraction","date":"2024-06-02","arxiv_id":"2406.00614","n_code_links":1,"syntology":null},{"paper":null,"title":"Global Rewards in Restless Multi-Armed Bandits","date":"2024-06-02","arxiv_id":"2406.00738","n_code_links":0,"syntology":null},{"paper":null,"title":"Extreme Value Monte Carlo Tree Search","date":"2024-05-28","arxiv_id":"2405.18248","n_code_links":0,"syntology":null},{"paper":"/paper/pytag-tabletop-games-for-multi-agent","title":"PyTAG: Tabletop Games for Multi-Agent Reinforcement Learning","date":"2024-05-28","arxiv_id":"2405.18123","n_code_links":1,"syntology":null},{"paper":"/paper/efficient-multi-agent-reinforcement-learning","title":"Efficient Multi-agent Reinforcement Learning by Planning","date":"2024-05-20","arxiv_id":"2405.11778","n_code_links":1,"syntology":null},{"paper":"/paper/value-augmented-sampling-for-language-model","title":"Value Augmented Sampling for Language Model Alignment and Personalization","date":"2024-05-10","arxiv_id":"2405.06639","n_code_links":1,"syntology":{"ran":1,"of":1,"unverified":0,"pointer_only":0}},{"paper":"/paper/rezero-boosting-mcts-based-algorithms-by-just","title":"ReZero: Boosting MCTS-based Algorithms by Backward-view and Entire-buffer Reanalyze","date":"2024-04-25","arxiv_id":"2404.16364","n_code_links":1,"syntology":null},{"paper":"/paper/tree-search-based-policy-optimization-under","title":"Tree Search-Based Policy Optimization under Stochastic Execution Delay","date":"2024-04-08","arxiv_id":"2404.05440","n_code_links":1,"syntology":{"ran":8,"of":8,"unverified":0,"pointer_only":8}},{"paper":null,"title":"Can Large Language Models Play Games? A Case Study of A Self-Play Approach","date":"2024-03-08","arxiv_id":"2403.05632","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":47},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":45},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":44},{"task":"/task/model-based-reinforcement-learning","name":"Model-based Reinforcement Learning","papers":21},{"task":"/task/decision-making","name":"Decision Making","papers":20},{"task":"/task/board-games","name":"Board Games","papers":16},{"task":"/task/deep-reinforcement-learning","name":"Deep Reinforcement Learning","papers":14},{"task":"/task/atari-games","name":"Atari Games","papers":12},{"task":"/task/continuous-control","name":"Continuous Control","papers":6},{"task":"/task/game-of-go","name":"Game of Go","papers":6},{"task":"/task/continuous-control","name":"continuous-control","papers":5},{"task":"/task/model","name":"model","papers":5},{"task":"/task/sequential-decision-making","name":"Sequential Decision Making","papers":4},{"task":"/task/atari-games-100k","name":"Atari Games 100k","papers":3},{"task":"/task/machine-learning","name":"BIG-bench Machine Learning","papers":3},{"task":"/task/bayesian-optimization","name":"Bayesian Optimization","papers":3},{"task":"/task/computational-efficiency","name":"Computational Efficiency","papers":3},{"task":"/task/general-reinforcement-learning","name":"General Reinforcement Learning","papers":3},{"task":"/task/heuristic-search","name":"Heuristic Search","papers":3},{"task":"/task/machine-translation","name":"Machine Translation","papers":3}],"tasks_shown":20,"n_tasks":111,"usage_by_year":[{"year":"2009","papers":1},{"year":"2010","papers":2},{"year":"2013","papers":2},{"year":"2014","papers":1},{"year":"2015","papers":1},{"year":"2017","papers":2},{"year":"2018","papers":8},{"year":"2019","papers":13},{"year":"2020","papers":29},{"year":"2021","papers":24},{"year":"2022","papers":21},{"year":"2023","papers":28},{"year":"2024","papers":23},{"year":"2025","papers":11}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/monte-carlo-tree-search"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}