{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-offline-evaluation-of-vision-based-driving","title":"On Offline Evaluation of Vision-based Driving Models","arxiv_id":"1809.04843","date":"2018-09-13","proceeding":"ECCV 2018 9","authors":["Felipe Codevilla","Antonio M. López","Vladlen Koltun","Alexey Dosovitskiy"],"abstract":"Autonomous driving models should ideally be evaluated by deploying them on a\nfleet of physical vehicles in the real world. Unfortunately, this approach is\nnot practical for the vast majority of researchers. An attractive alternative\nis to evaluate models offline, on a pre-collected validation dataset with\nground truth annotation. In this paper, we investigate the relation between\nvarious online and offline metrics for evaluation of autonomous driving models.\nWe find that offline prediction error is not necessarily correlated with\ndriving quality, and two models with identical prediction error can differ\ndramatically in their driving performance. We show that the correlation of\noffline evaluation with driving quality can be significantly improved by\nselecting an appropriate validation dataset and suitable offline metrics. The\nsupplementary video can be viewed at\nhttps://www.youtube.com/watch?v=P8K8Z-iF0cY","url_abs":"http://arxiv.org/abs/1809.04843v1","url_pdf":"http://arxiv.org/pdf/1809.04843v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-offline-evaluation-of-vision-based-driving","repo_url":"https://github.com/felipecode/coiltraine","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1809.04843","atlas_url":"https://app.syntology.ai/?focus=1809.04843","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}