{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/planning-for-sample-efficient-imitation","title":"Planning for Sample Efficient Imitation Learning","arxiv_id":"2210.09598","date":"2022-10-18","proceeding":null,"authors":["Zhao-Heng Yin","Weirui Ye","Qifeng Chen","Yang Gao"],"abstract":"Imitation learning is a class of promising policy learning algorithms that is free from many practical issues with reinforcement learning, such as the reward design issue and the exploration hardness. However, the current imitation algorithm struggles to achieve both high performance and high in-environment sample efficiency simultaneously. Behavioral Cloning (BC) does not need in-environment interactions, but it suffers from the covariate shift problem which harms its performance. Adversarial Imitation Learning (AIL) turns imitation learning into a distribution matching problem. It can achieve better performance on some tasks but it requires a large number of in-environment interactions. Inspired by the recent success of EfficientZero in RL, we propose EfficientImitate (EI), a planning-based imitation learning method that can achieve high in-environment sample efficiency and performance simultaneously. Our algorithmic contribution in this paper is two-fold. First, we extend AIL into the MCTS-based RL. Second, we show the seemingly incompatible two classes of imitation algorithms (BC and AIL) can be naturally unified under our framework, enjoying the benefits of both. We benchmark our method not only on the state-based DeepMind Control Suite, but also on the image version which many previous works find highly challenging. Experimental results show that EI achieves state-of-the-art results in performance and sample efficiency. EI shows over 4x gain in performance in the limited sample setting on state-based and image-based tasks and can solve challenging problems like Humanoid, where previous methods fail with small amount of interactions. Our code is available at https://github.com/zhaohengyin/EfficientImitate.","url_abs":"https://arxiv.org/abs/2210.09598v1","url_pdf":"https://arxiv.org/pdf/2210.09598v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"planning-for-sample-efficient-imitation","repo_url":"https://github.com/zhaohengyin/EfficientImitate","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"imitation-learning","task_name":"Imitation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.09598","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.09598"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/zhaohengyin/EfficientImitate","reach":null}],"summary":{"ran":4,"ran_draft_wrong":2,"ran_fixture":1,"ran_honours":1,"unverified":2},"by_repo_kind":{"official":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"89159fd394eef8da","entry":"ActorCriticNetwork","repo":"zhaohengyin/EfficientImitate","repo_kind":"official","path":"algo/obs_ei/model.py","file_url":"https://github.com/zhaohengyin/EfficientImitate/blob/HEAD/algo/obs_ei/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"89159fd394eef8da"}},{"code_sha256_prefix":"d69e09cf8b1b9c27","entry":"PredNetwork","repo":"zhaohengyin/EfficientImitate","repo_kind":"official","path":"algo/obs_ei/model.py","file_url":"https://github.com/zhaohengyin/EfficientImitate/blob/HEAD/algo/obs_ei/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d69e09cf8b1b9c27"}},{"code_sha256_prefix":"09ca94f5f2b3ffda","entry":"RepresentationNetwork","repo":"zhaohengyin/EfficientImitate","repo_kind":"official","path":"algo/obs_ei/model.py","file_url":"https://github.com/zhaohengyin/EfficientImitate/blob/HEAD/algo/obs_ei/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"09ca94f5f2b3ffda"}},{"code_sha256_prefix":"cacbaaa63c29f934","entry":"RewardNetwork","repo":"zhaohengyin/EfficientImitate","repo_kind":"official","path":"algo/obs_ei/model.py","file_url":"https://github.com/zhaohengyin/EfficientImitate/blob/HEAD/algo/obs_ei/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cacbaaa63c29f934"}},{"code_sha256_prefix":"88b04eafd87435f2","entry":"dict_to_cpu","repo":"zhaohengyin/EfficientImitate","repo_kind":"official","path":"algo/obs_ei/model.py","file_url":"https://github.com/zhaohengyin/EfficientImitate/blob/HEAD/algo/obs_ei/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"88b04eafd87435f2"}},{"code_sha256_prefix":"4c8aa25db20d758e","entry":"mlp","repo":"zhaohengyin/EfficientImitate","repo_kind":"official","path":"algo/obs_ei/model.py","file_url":"https://github.com/zhaohengyin/EfficientImitate/blob/HEAD/algo/obs_ei/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c8aa25db20d758e"}},{"code_sha256_prefix":"8a376431bfa31f3a","entry":"support_to_scalar","repo":"zhaohengyin/EfficientImitate","repo_kind":"official","path":"algo/obs_ei/model.py","file_url":"https://github.com/zhaohengyin/EfficientImitate/blob/HEAD/algo/obs_ei/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8a376431bfa31f3a"}},{"code_sha256_prefix":"255167c323c80d7b","entry":"tensor_to_numpy","repo":"zhaohengyin/EfficientImitate","repo_kind":"official","path":"algo/obs_ei/model.py","file_url":"https://github.com/zhaohengyin/EfficientImitate/blob/HEAD/algo/obs_ei/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"255167c323c80d7b"}},{"code_sha256_prefix":"a2ab73a0251612a4","entry":"AbstractNetwork","repo":"zhaohengyin/EfficientImitate","repo_kind":"official","path":"algo/obs_ei/model.py","file_url":"https://github.com/zhaohengyin/EfficientImitate/blob/HEAD/algo/obs_ei/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a2ab73a0251612a4"}},{"code_sha256_prefix":"184612793da7dac1","entry":"MuZeroResidualNetwork","repo":"zhaohengyin/EfficientImitate","repo_kind":"official","path":"algo/obs_ei/model.py","file_url":"https://github.com/zhaohengyin/EfficientImitate/blob/HEAD/algo/obs_ei/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"184612793da7dac1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}