{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/3d-visual-perception-for-self-driving-cars","title":"3D Visual Perception for Self-Driving Cars using a Multi-Camera System: Calibration, Mapping, Localization, and Obstacle Detection","arxiv_id":"1708.09839","date":"2017-08-31","proceeding":null,"authors":["Christian Häne","Lionel Heng","Gim Hee Lee","Friedrich Fraundorfer","Paul Furgale","Torsten Sattler","Marc Pollefeys"],"abstract":"Cameras are a crucial exteroceptive sensor for self-driving cars as they are\nlow-cost and small, provide appearance information about the environment, and\nwork in various weather conditions. They can be used for multiple purposes such\nas visual navigation and obstacle detection. We can use a surround multi-camera\nsystem to cover the full 360-degree field-of-view around the car. In this way,\nwe avoid blind spots which can otherwise lead to accidents. To minimize the\nnumber of cameras needed for surround perception, we utilize fisheye cameras.\nConsequently, standard vision pipelines for 3D mapping, visual localization,\nobstacle detection, etc. need to be adapted to take full advantage of the\navailability of multiple cameras rather than treat each camera individually. In\naddition, processing of fisheye images has to be supported. In this paper, we\ndescribe the camera calibration and subsequent processing pipeline for\nmulti-fisheye-camera systems developed as part of the V-Charge project. This\nproject seeks to enable automated valet parking for self-driving cars. Our\npipeline is able to precisely calibrate multi-camera systems, build sparse 3D\nmaps for visual navigation, visually localize the car with respect to these\nmaps, generate accurate dense maps, as well as detect obstacles based on\nreal-time depth map extraction.","url_abs":"http://arxiv.org/abs/1708.09839v1","url_pdf":"http://arxiv.org/pdf/1708.09839v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"3d-visual-perception-for-self-driving-cars","repo_url":"https://github.com/hengli/camodocal","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"camera-calibration","task_name":"Camera Calibration"},{"task_slug":"self-driving-cars","task_name":"Self-Driving Cars"},{"task_slug":"visual-localization","task_name":"Visual Localization"},{"task_slug":"visual-navigation","task_name":"Visual Navigation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1708.09839","atlas_url":"https://app.syntology.ai/?focus=1708.09839","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}