{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/interpretable-end-to-end-urban-autonomous","title":"Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learning","arxiv_id":"2001.08726","date":"2020-01-23","proceeding":null,"authors":["Jianyu Chen","Shengbo Eben Li","Masayoshi Tomizuka"],"abstract":"Unlike popular modularized framework, end-to-end autonomous driving seeks to solve the perception, decision and control problems in an integrated way, which can be more adapting to new scenarios and easier to generalize at scale. However, existing end-to-end approaches are often lack of interpretability, and can only deal with simple driving tasks like lane keeping. In this paper, we propose an interpretable deep reinforcement learning method for end-to-end autonomous driving, which is able to handle complex urban scenarios. A sequential latent environment model is introduced and learned jointly with the reinforcement learning process. With this latent model, a semantic birdeye mask can be generated, which is enforced to connect with a certain intermediate property in today's modularized framework for the purpose of explaining the behaviors of learned policy. The latent space also significantly reduces the sample complexity of reinforcement learning. Comparison tests with a simulated autonomous car in CARLA show that the performance of our method in urban scenarios with crowded surrounding vehicles dominates many baselines including DQN, DDPG, TD3 and SAC. Moreover, through masked outputs, the learned policy is able to provide a better explanation of how the car reasons about the driving environment. The codes and videos of this work are available at our github repo and project website.","url_abs":"https://arxiv.org/abs/2001.08726v3","url_pdf":"https://arxiv.org/pdf/2001.08726v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"interpretable-end-to-end-urban-autonomous","repo_url":"https://github.com/cjy1992/interp-e2e-driving","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"interpretable-end-to-end-urban-autonomous","repo_url":"https://github.com/cjy1992/gym-carla","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"interpretable-end-to-end-urban-autonomous","repo_url":"https://github.com/samadiamir/IE2E","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"interpretable-end-to-end-urban-autonomous","repo_url":"https://github.com/xzxzxzxz/interp","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"carla","method_name":"CARLA"},{"method_slug":"clipped-double-q-learning","method_name":"Clipped Double Q-learning"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"ddpg","method_name":"DDPG"},{"method_slug":"dqn","method_name":"DQN"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"},{"method_slug":"experience-replay","method_name":"Experience Replay"},{"method_slug":"ppo","method_name":"PPO"},{"method_slug":"q-learning","method_name":"Q-Learning"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"td3","method_name":"TD3"},{"method_slug":"target-policy-smoothing","method_name":"Target Policy Smoothing"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2001.08726","atlas_url":"https://app.syntology.ai/?focus=2001.08726","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2001.08726"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cjy1992/interp-e2e-driving","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cjy1992/gym-carla","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/samadiamir/IE2E","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xzxzxzxz/interp","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"unverified":4},"by_repo_kind":{"official":{"samples":6,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5f76afd9ff3e3df7","entry":"get_latent_reconstruction_videos","repo":"cjy1992/interp-e2e-driving","repo_kind":"official","path":"train_eval.py","file_url":"https://github.com/cjy1992/interp-e2e-driving/blob/HEAD/train_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5f76afd9ff3e3df7"}},{"code_sha256_prefix":"e119c06a408d8401","entry":"pad_and_concatenate_videos","repo":"cjy1992/interp-e2e-driving","repo_kind":"official","path":"train_eval.py","file_url":"https://github.com/cjy1992/interp-e2e-driving/blob/HEAD/train_eval.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e119c06a408d8401"}},{"code_sha256_prefix":"93dde1403b38e053","entry":"encode_gif","repo":"cjy1992/interp-e2e-driving","repo_kind":"official","path":"interp_e2e_driving/utils/gif_utils.py","file_url":"https://github.com/cjy1992/interp-e2e-driving/blob/HEAD/interp_e2e_driving/utils/gif_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"93dde1403b38e053"}},{"code_sha256_prefix":"e96e68925ab5c1d1","entry":"gif_summary_v2","repo":"cjy1992/interp-e2e-driving","repo_kind":"official","path":"interp_e2e_driving/utils/gif_utils.py","file_url":"https://github.com/cjy1992/interp-e2e-driving/blob/HEAD/interp_e2e_driving/utils/gif_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e96e68925ab5c1d1"}},{"code_sha256_prefix":"a62c46c13fc9bf7f","entry":"map_distribution_structure","repo":"cjy1992/interp-e2e-driving","repo_kind":"official","path":"interp_e2e_driving/utils/nest_utils.py","file_url":"https://github.com/cjy1992/interp-e2e-driving/blob/HEAD/interp_e2e_driving/utils/nest_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a62c46c13fc9bf7f"}},{"code_sha256_prefix":"b94b1990bf07f859","entry":"py_gif_summary","repo":"cjy1992/interp-e2e-driving","repo_kind":"official","path":"interp_e2e_driving/utils/gif_utils.py","file_url":"https://github.com/cjy1992/interp-e2e-driving/blob/HEAD/interp_e2e_driving/utils/gif_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b94b1990bf07f859"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}