{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/end-to-end-autonomous-driving-perception-with","title":"End-to-end Autonomous Driving Perception with Sequential Latent Representation Learning","arxiv_id":"2003.12464","date":"2020-03-21","proceeding":null,"authors":["Jianyu Chen","Zhuo Xu","Masayoshi Tomizuka"],"abstract":"Current autonomous driving systems are composed of a perception system and a decision system. Both of them are divided into multiple subsystems built up with lots of human heuristics. An end-to-end approach might clean up the system and avoid huge efforts of human engineering, as well as obtain better performance with increasing data and computation resources. Compared to the decision system, the perception system is more suitable to be designed in an end-to-end framework, since it does not require online driving exploration. In this paper, we propose a novel end-to-end approach for autonomous driving perception. A latent space is introduced to capture all relevant features useful for perception, which is learned through sequential latent representation learning. The learned end-to-end perception model is able to solve the detection, tracking, localization and mapping problems altogether with only minimum human engineering efforts and without storing any maps online. The proposed method is evaluated in a realistic urban driving simulator, with both camera image and lidar point cloud as sensor inputs. The codes and videos of this work are available at our github repo and project website.","url_abs":"https://arxiv.org/abs/2003.12464v2","url_pdf":"https://arxiv.org/pdf/2003.12464v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"end-to-end-autonomous-driving-perception-with","repo_url":"https://github.com/cjy1992/detect-loc-map","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"end-to-end-autonomous-driving-perception-with","repo_url":"https://github.com/cjy1992/gym-carla","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2003.12464","atlas_url":"https://app.syntology.ai/?focus=2003.12464","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2003.12464"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cjy1992/gym-carla","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cjy1992/detect-loc-map","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":8},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1},"listed":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b01f5dce8e560352","entry":"compute_connection","repo":"cjy1992/gym-carla","repo_kind":"listed","path":"gym_carla/envs/route_planner.py","file_url":"https://github.com/cjy1992/gym-carla/blob/HEAD/gym_carla/envs/route_planner.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b01f5dce8e560352"}},{"code_sha256_prefix":"67e87c930497a29b","entry":"focal_loss","repo":"cjy1992/detect-loc-map","repo_kind":"official","path":"perception_driving/networks/sequential_latent_pixor_network.py","file_url":"https://github.com/cjy1992/detect-loc-map/blob/HEAD/perception_driving/networks/sequential_latent_pixor_network.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"67e87c930497a29b"}},{"code_sha256_prefix":"3b06cf6170c8f79a","entry":"get_info","repo":"cjy1992/gym-carla","repo_kind":"listed","path":"gym_carla/envs/misc.py","file_url":"https://github.com/cjy1992/gym-carla/blob/HEAD/gym_carla/envs/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b06cf6170c8f79a"}},{"code_sha256_prefix":"48de9d3f6175f0d0","entry":"get_pos","repo":"cjy1992/gym-carla","repo_kind":"listed","path":"gym_carla/envs/misc.py","file_url":"https://github.com/cjy1992/gym-carla/blob/HEAD/gym_carla/envs/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"48de9d3f6175f0d0"}},{"code_sha256_prefix":"3b98ce0d762ff08e","entry":"get_speed","repo":"cjy1992/gym-carla","repo_kind":"listed","path":"gym_carla/envs/misc.py","file_url":"https://github.com/cjy1992/gym-carla/blob/HEAD/gym_carla/envs/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b98ce0d762ff08e"}},{"code_sha256_prefix":"e119c06a408d8401","entry":"pad_and_concatenate_videos","repo":"cjy1992/detect-loc-map","repo_kind":"official","path":"train_eval.py","file_url":"https://github.com/cjy1992/detect-loc-map/blob/HEAD/train_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e119c06a408d8401"}},{"code_sha256_prefix":"a6ee22444d90d1b0","entry":"retrieve_options","repo":"cjy1992/gym-carla","repo_kind":"listed","path":"gym_carla/envs/route_planner.py","file_url":"https://github.com/cjy1992/gym-carla/blob/HEAD/gym_carla/envs/route_planner.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a6ee22444d90d1b0"}},{"code_sha256_prefix":"9cbbe17e943a3794","entry":"smoothL1_loss","repo":"cjy1992/detect-loc-map","repo_kind":"official","path":"perception_driving/networks/sequential_latent_pixor_network.py","file_url":"https://github.com/cjy1992/detect-loc-map/blob/HEAD/perception_driving/networks/sequential_latent_pixor_network.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9cbbe17e943a3794"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}