{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/failing-to-learn-autonomously-identifying","title":"Failing to Learn: Autonomously Identifying Perception Failures for Self-driving Cars","arxiv_id":"1707.00051","date":"2017-06-30","proceeding":null,"authors":["Manikandasriram Srinivasan Ramanagopal","Cyrus Anderson","Ram Vasudevan","Matthew Johnson-Roberson"],"abstract":"One of the major open challenges in self-driving cars is the ability to\ndetect cars and pedestrians to safely navigate in the world. Deep\nlearning-based object detector approaches have enabled great advances in using\ncamera imagery to detect and classify objects. But for a safety critical\napplication, such as autonomous driving, the error rates of the current state\nof the art are still too high to enable safe operation. Moreover, the\ncharacterization of object detector performance is primarily limited to testing\non prerecorded datasets. Errors that occur on novel data go undetected without\nadditional human labels. In this letter, we propose an automated method to\nidentify mistakes made by object detectors without ground truth labels. We show\nthat inconsistencies in the object detector output between a pair of similar\nimages can be used as hypotheses for false negatives (e.g., missed detections)\nand using a novel set of features for each hypothesis, an off-the-shelf binary\nclassifier can be used to find valid errors. In particular, we study two\ndistinct cues - temporal and stereo inconsistencies - using data that are\nreadily available on most autonomous vehicles. Our method can be used with any\ncamera-based object detector and we illustrate the technique on several sets of\nreal world data. We show that a state-of-the-art detector, tracker, and our\nclassifier trained only on synthetic data can identify valid errors on KITTI\ntracking dataset with an average precision of 0.94. We also release a new\ntracking dataset with 104 sequences totaling 80,655 labeled pairs of stereo\nimages along with ground truth disparity from a game engine to facilitate\nfurther research. The dataset and code are available at\nhttps://fcav.engin.umich.edu/research/failing-to-learn","url_abs":"http://arxiv.org/abs/1707.00051v4","url_pdf":"http://arxiv.org/pdf/1707.00051v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"failing-to-learn-autonomously-identifying","repo_url":"https://github.com/umautobots/failing-to-learn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"},{"task_slug":"autonomous-vehicles","task_name":"Autonomous Vehicles"},{"task_slug":"navigate","task_name":"Navigate"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"self-driving-cars","task_name":"Self-Driving Cars"},{"task_slug":null,"task_name":"valid"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1707.00051","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}