{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/non-adversarial-imitation-learning-and-its","title":"Non-Adversarial Imitation Learning and its Connections to Adversarial Methods","arxiv_id":"2008.03525","date":"2020-08-08","proceeding":null,"authors":["Oleg Arenz","Gerhard Neumann"],"abstract":"Many modern methods for imitation learning and inverse reinforcement learning, such as GAIL or AIRL, are based on an adversarial formulation. These methods apply GANs to match the expert's distribution over states and actions with the implicit state-action distribution induced by the agent's policy. However, by framing imitation learning as a saddle point problem, adversarial methods can suffer from unstable optimization, and convergence can only be shown for small policy updates. We address these problems by proposing a framework for non-adversarial imitation learning. The resulting algorithms are similar to their adversarial counterparts and, thus, provide insights for adversarial imitation learning methods. Most notably, we show that AIRL is an instance of our non-adversarial formulation, which enables us to greatly simplify its derivations and obtain stronger convergence guarantees. We also show that our non-adversarial formulation can be used to derive novel algorithms by presenting a method for offline imitation learning that is inspired by the recent ValueDice algorithm, but does not rely on small policy updates for convergence. In our simulated robot experiments, our offline method for non-adversarial imitation learning seems to perform best when using many updates for policy and discriminator at each iteration and outperforms behavioral cloning and ValueDice.","url_abs":"https://arxiv.org/abs/2008.03525v1","url_pdf":"https://arxiv.org/pdf/2008.03525v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"non-adversarial-imitation-learning-and-its","repo_url":"https://github.com/OlegArenz/O-NAIL","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"imitation-learning","task_name":"Imitation Learning"}],"methods":[{"method_slug":"ail","method_name":"GAIL"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2008.03525","atlas_url":"https://app.syntology.ai/?focus=2008.03525","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2008.03525"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/OlegArenz/O-NAIL","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b1f9cebf0d177140","entry":"boolean_string","repo":"OlegArenz/O-NAIL","repo_kind":"official","path":"common/utlis.py","file_url":"https://github.com/OlegArenz/O-NAIL/blob/HEAD/common/utlis.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1f9cebf0d177140"}},{"code_sha256_prefix":"7ab6586c15485b41","entry":"build_kwargs_from_command_line_args","repo":"OlegArenz/O-NAIL","repo_kind":"official","path":"run_experiment.py","file_url":"https://github.com/OlegArenz/O-NAIL/blob/HEAD/run_experiment.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ab6586c15485b41"}},{"code_sha256_prefix":"7f3c670f6150299c","entry":"process_trajectories","repo":"OlegArenz/O-NAIL","repo_kind":"official","path":"common/utlis.py","file_url":"https://github.com/OlegArenz/O-NAIL/blob/HEAD/common/utlis.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f3c670f6150299c"}},{"code_sha256_prefix":"22989ac3bf03e8a8","entry":"roll_out_policy","repo":"OlegArenz/O-NAIL","repo_kind":"official","path":"common/utlis.py","file_url":"https://github.com/OlegArenz/O-NAIL/blob/HEAD/common/utlis.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"22989ac3bf03e8a8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}