{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/online-abstraction-with-mdp-homomorphisms-for","title":"Online Abstraction with MDP Homomorphisms for Deep Learning","arxiv_id":"1811.12929","date":"2018-11-30","proceeding":null,"authors":["Ondrej Biza","Robert Platt"],"abstract":"Abstraction of Markov Decision Processes is a useful tool for solving complex\nproblems, as it can ignore unimportant aspects of an environment, simplifying\nthe process of learning an optimal policy. In this paper, we propose a new\nalgorithm for finding abstract MDPs in environments with continuous state\nspaces. It is based on MDP homomorphisms, a structure-preserving mapping\nbetween MDPs. We demonstrate our algorithm's ability to learn abstractions from\ncollected experience and show how to reuse the abstractions to guide\nexploration in new tasks the agent encounters. Our novel task transfer method\noutperforms baselines based on a deep Q-network in the majority of our\nexperiments. The source code is at https://github.com/ondrejba/aamas_19.","url_abs":"http://arxiv.org/abs/1811.12929v2","url_pdf":"http://arxiv.org/pdf/1811.12929v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"online-abstraction-with-mdp-homomorphisms-for","repo_url":"https://github.com/ondrejba/aamas_19","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1811.12929","atlas_url":"https://app.syntology.ai/?focus=1811.12929","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}