{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/model-primitive-hierarchical-lifelong","title":"Model Primitive Hierarchical Lifelong Reinforcement Learning","arxiv_id":"1903.01567","date":"2019-03-04","proceeding":null,"authors":["Bohan Wu","Jayesh K. Gupta","Mykel J. Kochenderfer"],"abstract":"Learning interpretable and transferable subpolicies and performing task\ndecomposition from a single, complex task is difficult. Some traditional\nhierarchical reinforcement learning techniques enforce this decomposition in a\ntop-down manner, while meta-learning techniques require a task distribution at\nhand to learn such decompositions. This paper presents a framework for using\ndiverse suboptimal world models to decompose complex task solutions into\nsimpler modular subpolicies. This framework performs automatic decomposition of\na single source task in a bottom up manner, concurrently learning the required\nmodular subpolicies as well as a controller to coordinate them. We perform a\nseries of experiments on high dimensional continuous action control tasks to\ndemonstrate the effectiveness of this approach at both complex single task\nlearning and lifelong learning. Finally, we perform ablation studies to\nunderstand the importance and robustness of different elements in the framework\nand limitations to this approach.","url_abs":"http://arxiv.org/abs/1903.01567v1","url_pdf":"http://arxiv.org/pdf/1903.01567v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"model-primitive-hierarchical-lifelong","repo_url":"https://github.com/sisl/MPHRL","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"hierarchical-reinforcement-learning","task_name":"Hierarchical Reinforcement Learning"},{"task_slug":"lifelong-learning","task_name":"Lifelong learning"},{"task_slug":"meta-learning","task_name":"Meta-Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"model","task_name":"model"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1903.01567","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1903.01567"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sisl/MPHRL","reach":null}],"summary":{"ran_honours":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"daffe25a5d476485","entry":"value","repo":"sisl/MPHRL","repo_kind":"official","path":"config_model0.py","file_url":"https://github.com/sisl/MPHRL/blob/HEAD/config_model0.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"daffe25a5d476485"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}