{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mastering-2048-with-delayed-temporal","title":"Mastering 2048 with Delayed Temporal Coherence Learning, Multi-Stage Weight Promotion, Redundant Encoding and Carousel Shaping","arxiv_id":"1604.05085","date":"2016-04-18","proceeding":null,"authors":["Wojciech Jaśkowski"],"abstract":"2048 is an engaging single-player, nondeterministic video puzzle game, which,\nthanks to the simple rules and hard-to-master gameplay, has gained massive\npopularity in recent years. As 2048 can be conveniently embedded into the\ndiscrete-state Markov decision processes framework, we treat it as a testbed\nfor evaluating existing and new methods in reinforcement learning. With the aim\nto develop a strong 2048 playing program, we employ temporal difference\nlearning with systematic n-tuple networks. We show that this basic method can\nbe significantly improved with temporal coherence learning, multi-stage\nfunction approximator with weight promotion, carousel shaping, and redundant\nencoding. In addition, we demonstrate how to take advantage of the\ncharacteristics of the n-tuple network, to improve the algorithmic\neffectiveness of the learning process by i) delaying the (decayed) update and\napplying lock-free optimistic parallelism to effortlessly make advantage of\nmultiple CPU cores. This way, we were able to develop the best known 2048\nplaying program to date, which confirms the effectiveness of the introduced\nmethods for discrete-state Markov decision problems.","url_abs":"http://arxiv.org/abs/1604.05085v3","url_pdf":"http://arxiv.org/pdf/1604.05085v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mastering-2048-with-delayed-temporal","repo_url":"https://github.com/aszczepanski/2048","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"mastering-2048-with-delayed-temporal","repo_url":"https://github.com/wjaskowski/mastering-2048","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"mastering-2048-with-delayed-temporal","repo_url":"https://github.com/abachurin/2048","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"mastering-2048-with-delayed-temporal","repo_url":"https://github.com/thomasahle/mcts-2048","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":null,"task_name":"CPU"},{"task_slug":"2048","task_name":"Playing the Game of 2048"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}