{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/energy-based-imitation-learning","title":"Energy-Based Imitation Learning","arxiv_id":"2004.09395","date":"2020-04-20","proceeding":null,"authors":["Minghuan Liu","Tairan He","Minkai Xu","Wei-Nan Zhang"],"abstract":"We tackle a common scenario in imitation learning (IL), where agents try to recover the optimal policy from expert demonstrations without further access to the expert or environment reward signals. Except the simple Behavior Cloning (BC) that adopts supervised learning followed by the problem of compounding error, previous solutions like inverse reinforcement learning (IRL) and recent generative adversarial methods involve a bi-level or alternating optimization for updating the reward function and the policy, suffering from high computational cost and training instability. Inspired by recent progress in energy-based model (EBM), in this paper, we propose a simplified IL framework named Energy-Based Imitation Learning (EBIL). Instead of updating the reward and policy iteratively, EBIL breaks out of the traditional IRL paradigm by a simple and flexible two-stage solution: first estimating the expert energy as the surrogate reward function through score matching, then utilizing such a reward for learning the policy by reinforcement learning algorithms. EBIL combines the idea of both EBM and occupancy measure matching, and via theoretic analysis we reveal that EBIL and Max-Entropy IRL (MaxEnt IRL) approaches are two sides of the same coin, and thus EBIL could be an alternative of adversarial IRL methods. Extensive experiments on qualitative and quantitative evaluations indicate that EBIL is able to recover meaningful and interpretative reward signals while achieving effective and comparable performance against existing algorithms on IL benchmarks.","url_abs":"https://arxiv.org/abs/2004.09395v4","url_pdf":"https://arxiv.org/pdf/2004.09395v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"energy-based-imitation-learning","repo_url":"https://github.com/apexrl/EBIL-torch","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"imitation-learning","task_name":"Imitation Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"ebm","method_name":"EBM"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2004.09395","atlas_url":"https://app.syntology.ai/?focus=2004.09395","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2004.09395"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/apexrl/EBIL-torch","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c22a3eb163c4e832","entry":"concat_trajs","repo":"apexrl/EBIL-torch","repo_kind":"official","path":"rlkit/torch/ebil/bc.py","file_url":"https://github.com/apexrl/EBIL-torch/blob/HEAD/rlkit/torch/ebil/bc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c22a3eb163c4e832"}},{"code_sha256_prefix":"f97c59ae63058414","entry":"reward_func","repo":"apexrl/EBIL-torch","repo_kind":"official","path":"rlkit/torch/ebil/ebil.py","file_url":"https://github.com/apexrl/EBIL-torch/blob/HEAD/rlkit/torch/ebil/ebil.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f97c59ae63058414"}},{"code_sha256_prefix":"2fcee7ba769344dd","entry":"sum_aggregator","repo":"apexrl/EBIL-torch","repo_kind":"official","path":"rlkit/torch/ebil/encoders/aggregators.py","file_url":"https://github.com/apexrl/EBIL-torch/blob/HEAD/rlkit/torch/ebil/encoders/aggregators.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2fcee7ba769344dd"}},{"code_sha256_prefix":"c82c1656a9e07512","entry":"tanh_sum_aggregator","repo":"apexrl/EBIL-torch","repo_kind":"official","path":"rlkit/torch/ebil/encoders/aggregators.py","file_url":"https://github.com/apexrl/EBIL-torch/blob/HEAD/rlkit/torch/ebil/encoders/aggregators.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c82c1656a9e07512"}},{"code_sha256_prefix":"1a2244e58b6aae79","entry":"mean_aggregator","repo":"apexrl/EBIL-torch","repo_kind":"official","path":"rlkit/torch/ebil/encoders/aggregators.py","file_url":"https://github.com/apexrl/EBIL-torch/blob/HEAD/rlkit/torch/ebil/encoders/aggregators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a2244e58b6aae79"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}