{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lightemma-lightweight-end-to-end-multimodal","title":"LightEMMA: Lightweight End-to-End Multimodal Model for Autonomous Driving","arxiv_id":"2505.00284","date":"2025-05-01","proceeding":null,"authors":["Zhijie Qiao","Haowei Li","Zhong Cao","Henry X. Liu"],"abstract":"Vision-Language Models (VLMs) have demonstrated significant potential for end-to-end autonomous driving. However, fully exploiting their capabilities for safe and reliable vehicle control remains an open research challenge. To systematically examine advances and limitations of VLMs in driving tasks, we introduce LightEMMA, a Lightweight End-to-End Multimodal Model for Autonomous driving. LightEMMA provides a unified, VLM-based autonomous driving framework without ad hoc customizations, enabling easy integration and evaluation of evolving state-of-the-art commercial and open-source models. We construct twelve autonomous driving agents using various VLMs and evaluate their performance on the nuScenes prediction task, comprehensively assessing metrics such as inference time, computational cost, and predictive accuracy. Illustrative examples highlight that, despite their strong scenario interpretation capabilities, VLMs' practical performance in autonomous driving tasks remains concerning, emphasizing the need for further improvements. The code is available at https://github.com/michigan-traffic-lab/LightEMMA.","url_abs":"https://arxiv.org/abs/2505.00284v1","url_pdf":"https://arxiv.org/pdf/2505.00284v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lightemma-lightweight-end-to-end-multimodal","repo_url":"https://github.com/michigan-traffic-lab/lightemma","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"}],"methods":[{"method_slug":"hoc","method_name":"HOC"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2505.00284","atlas_url":"https://app.syntology.ai/?focus=2505.00284","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.00284"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/michigan-traffic-lab/lightemma","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"unverified":3},"by_repo_kind":{"official":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a29c978a36cd6592","entry":"extract_driving_action","repo":"michigan-traffic-lab/lightemma","repo_kind":"official","path":"utils.py","file_url":"https://github.com/michigan-traffic-lab/lightemma/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a29c978a36cd6592"}},{"code_sha256_prefix":"02cec97f65c5fe5b","entry":"is_valid_action","repo":"michigan-traffic-lab/lightemma","repo_kind":"official","path":"utils.py","file_url":"https://github.com/michigan-traffic-lab/lightemma/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"02cec97f65c5fe5b"}},{"code_sha256_prefix":"d9f178a6037e10af","entry":"quaternion_to_yaw","repo":"michigan-traffic-lab/lightemma","repo_kind":"official","path":"utils.py","file_url":"https://github.com/michigan-traffic-lab/lightemma/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9f178a6037e10af"}},{"code_sha256_prefix":"7cd9ae59300658e2","entry":"collect_errors","repo":"michigan-traffic-lab/lightemma","repo_kind":"official","path":"evaluate_all.py","file_url":"https://github.com/michigan-traffic-lab/lightemma/blob/HEAD/evaluate_all.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7cd9ae59300658e2"}},{"code_sha256_prefix":"8bc72f3d51539799","entry":"evaluate","repo":"michigan-traffic-lab/lightemma","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/michigan-traffic-lab/lightemma/blob/HEAD/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8bc72f3d51539799"}},{"code_sha256_prefix":"5029d05f539396ce","entry":"evaluate","repo":"michigan-traffic-lab/lightemma","repo_kind":"official","path":"evaluate_all.py","file_url":"https://github.com/michigan-traffic-lab/lightemma/blob/HEAD/evaluate_all.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5029d05f539396ce"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}