{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/opendrivevla-towards-end-to-end-autonomous","title":"OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model","arxiv_id":"2503.23463","date":"2025-03-30","proceeding":null,"authors":["Xingcheng Zhou","Xuyuan Han","Feng Yang","Yunpu Ma","Alois C. Knoll"],"abstract":"We present OpenDriveVLA, a Vision-Language Action (VLA) model designed for end-to-end autonomous driving. OpenDriveVLA builds upon open-source pre-trained large Vision-Language Models (VLMs) to generate reliable driving actions, conditioned on 3D environmental perception, ego vehicle states, and driver commands. To bridge the modality gap between driving visual representations and language embeddings, we propose a hierarchical vision-language alignment process, projecting both 2D and 3D structured visual tokens into a unified semantic space. Besides, OpenDriveVLA models the dynamic relationships between the ego vehicle, surrounding agents, and static road elements through an autoregressive agent-env-ego interaction process, ensuring both spatially and behaviorally informed trajectory planning. Extensive experiments on the nuScenes dataset demonstrate that OpenDriveVLA achieves state-of-the-art results across open-loop trajectory planning and driving-related question-answering tasks. Qualitative analyses further illustrate OpenDriveVLA's superior capability to follow high-level driving commands and robustly generate trajectories under challenging scenarios, highlighting its potential for next-generation end-to-end autonomous driving. We will release our code to facilitate further research in this domain.","url_abs":"https://arxiv.org/abs/2503.23463v1","url_pdf":"https://arxiv.org/pdf/2503.23463v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"opendrivevla-towards-end-to-end-autonomous","repo_url":"https://github.com/DriveVLA/OpenDriveVLA","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"},{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"motion-planning","task_name":"Motion Planning"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"},{"task_slug":"trajectory-planning","task_name":"Trajectory Planning"},{"task_slug":"vision-language-action","task_name":"Vision-Language-Action"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.23463","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.23463"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/DriveVLA/OpenDriveVLA","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_fixture":2,"unverified":7},"by_repo_kind":{"official":{"samples":9,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b0a851cae92754ff","entry":"extract_patches","repo":"DriveVLA/OpenDriveVLA","repo_kind":"official","path":"llava/mm_utils.py","file_url":"https://github.com/DriveVLA/OpenDriveVLA/blob/HEAD/llava/mm_utils.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b0a851cae92754ff"}},{"code_sha256_prefix":"7606525af238fb64","entry":"unpad_image","repo":"DriveVLA/OpenDriveVLA","repo_kind":"official","path":"llava/model/llava_arch.py","file_url":"https://github.com/DriveVLA/OpenDriveVLA/blob/HEAD/llava/model/llava_arch.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7606525af238fb64"}},{"code_sha256_prefix":"3c8823948724e269","entry":"auto_pad_images","repo":"DriveVLA/OpenDriveVLA","repo_kind":"official","path":"llava/mm_utils.py","file_url":"https://github.com/DriveVLA/OpenDriveVLA/blob/HEAD/llava/mm_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3c8823948724e269"}},{"code_sha256_prefix":"c8e18ac13ca83574","entry":"load_image","repo":"DriveVLA/OpenDriveVLA","repo_kind":"official","path":"drivevla/inference_drivevla.py","file_url":"https://github.com/DriveVLA/OpenDriveVLA/blob/HEAD/drivevla/inference_drivevla.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c8e18ac13ca83574"}},{"code_sha256_prefix":"f754b678dbeb412d","entry":"process_video_with_decord","repo":"DriveVLA/OpenDriveVLA","repo_kind":"official","path":"llava/utils.py","file_url":"https://github.com/DriveVLA/OpenDriveVLA/blob/HEAD/llava/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f754b678dbeb412d"}},{"code_sha256_prefix":"5bd0aa0a6397c421","entry":"process_video_with_pyav","repo":"DriveVLA/OpenDriveVLA","repo_kind":"official","path":"llava/utils.py","file_url":"https://github.com/DriveVLA/OpenDriveVLA/blob/HEAD/llava/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5bd0aa0a6397c421"}},{"code_sha256_prefix":"209ac10113b4c3cf","entry":"read_processed_ids","repo":"DriveVLA/OpenDriveVLA","repo_kind":"official","path":"drivevla/inference_drivevla.py","file_url":"https://github.com/DriveVLA/OpenDriveVLA/blob/HEAD/drivevla/inference_drivevla.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"209ac10113b4c3cf"}},{"code_sha256_prefix":"c80358dff96c25ce","entry":"resize_and_center_crop","repo":"DriveVLA/OpenDriveVLA","repo_kind":"official","path":"llava/mm_utils.py","file_url":"https://github.com/DriveVLA/OpenDriveVLA/blob/HEAD/llava/mm_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c80358dff96c25ce"}},{"code_sha256_prefix":"b20705b0dfd77ffc","entry":"safe_load_tokenizer","repo":"DriveVLA/OpenDriveVLA","repo_kind":"official","path":"llava/conversation.py","file_url":"https://github.com/DriveVLA/OpenDriveVLA/blob/HEAD/llava/conversation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b20705b0dfd77ffc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}