{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ordinal-monte-carlo-tree-search","title":"Ordinal Monte Carlo Tree Search","arxiv_id":"1901.04274","date":"2019-01-14","proceeding":null,"authors":["Tobias Joppen","Johannes Fürnkranz"],"abstract":"In many problem settings, most notably in game playing, an agent receives a\npossibly delayed reward for its actions. Often, those rewards are handcrafted\nand not naturally given. Even simple terminal-only rewards, like winning equals\n1 and losing equals -1, can not be seen as an unbiased statement, since these\nvalues are chosen arbitrarily, and the behavior of the learner may change with\ndifferent encodings, such as setting the value of a loss to -0:5, which is\noften done in practice to encourage learning. It is hard to argue about good\nrewards and the performance of an agent often depends on the design of the\nreward signal. In particular, in domains where states by nature only have an\nordinal ranking and where meaningful distance information between game state\nvalues are not available, a numerical reward signal is necessarily biased. In\nthis paper, we take a look at Monte Carlo Tree Search (MCTS), a popular\nalgorithm to solve MDPs, highlight a reoccurring problem concerning its use of\nrewards, and show that an ordinal treatment of the rewards overcomes this\nproblem. Using the General Video Game Playing framework we show a dominance of\nour newly proposed ordinal MCTS algorithm over preference-based MCTS, vanilla\nMCTS and various other MCTS variants.","url_abs":"http://arxiv.org/abs/1901.04274v1","url_pdf":"http://arxiv.org/pdf/1901.04274v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ordinal-monte-carlo-tree-search","repo_url":"https://github.com/coreylowman/ragz","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"ordinal-monte-carlo-tree-search","repo_url":"https://github.com/coreylowman/synthesis","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}