{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/offline-imitation-learning-with-variational","title":"Offline Imitation Learning with Variational Counterfactual Reasoning","arxiv_id":"2310.04706","date":"2023-10-07","proceeding":"NeurIPS 2023 11","authors":["Bowei He","Zexu Sun","Jinxin Liu","Shuai Zhang","Xu Chen","Chen Ma"],"abstract":"In offline imitation learning (IL), an agent aims to learn an optimal expert behavior policy without additional online environment interactions. However, in many real-world scenarios, such as robotics manipulation, the offline dataset is collected from suboptimal behaviors without rewards. Due to the scarce expert data, the agents usually suffer from simply memorizing poor trajectories and are vulnerable to variations in the environments, lacking the capability of generalizing to new environments. To automatically generate high-quality expert data and improve the generalization ability of the agent, we propose a framework named \\underline{O}ffline \\underline{I}mitation \\underline{L}earning with \\underline{C}ounterfactual data \\underline{A}ugmentation (OILCA) by doing counterfactual inference. In particular, we leverage identifiable variational autoencoder to generate \\textit{counterfactual} samples for expert data augmentation. We theoretically analyze the influence of the generated expert data and the improvement of generalization. Moreover, we conduct extensive experiments to demonstrate that our approach significantly outperforms various baselines on both \\textsc{DeepMind Control Suite} benchmark for in-distribution performance and \\textsc{CausalWorld} benchmark for out-of-distribution generalization. Our code is available at \\url{https://github.com/ZexuSun/OILCA-NeurIPS23}.","url_abs":"https://arxiv.org/abs/2310.04706v4","url_pdf":"https://arxiv.org/pdf/2310.04706v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"offline-imitation-learning-with-variational","repo_url":"https://github.com/zexusun/oilca-neurips23","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"counterfactual-inference","task_name":"Counterfactual Inference"},{"task_slug":"counterfactual-reasoning","task_name":"Counterfactual Reasoning"},{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"imitation-learning","task_name":"Imitation Learning"},{"task_slug":"out-of-distribution-generalization","task_name":"Out-of-Distribution Generalization"},{"task_slug":null,"task_name":"counterfactual"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.04706","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}