{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lidar-camera-fusion-for-road-detection-using","title":"LIDAR-Camera Fusion for Road Detection Using Fully Convolutional Neural Networks","arxiv_id":"1809.07941","date":"2018-09-21","proceeding":null,"authors":["Luca Caltagirone","Mauro Bellone","Lennart Svensson","Mattias Wahde"],"abstract":"In this work, a deep learning approach has been developed to carry out road\ndetection by fusing LIDAR point clouds and camera images. An unstructured and\nsparse point cloud is first projected onto the camera image plane and then\nupsampled to obtain a set of dense 2D images encoding spatial information.\nSeveral fully convolutional neural networks (FCNs) are then trained to carry\nout road detection, either by using data from a single sensor, or by using\nthree fusion strategies: early, late, and the newly proposed cross fusion.\nWhereas in the former two fusion approaches, the integration of multimodal\ninformation is carried out at a predefined depth level, the cross fusion FCN is\ndesigned to directly learn from data where to integrate information; this is\naccomplished by using trainable cross connections between the LIDAR and the\ncamera processing branches.\n  To further highlight the benefits of using a multimodal system for road\ndetection, a data set consisting of visually challenging scenes was extracted\nfrom driving sequences of the KITTI raw data set. It was then demonstrated\nthat, as expected, a purely camera-based FCN severely underperforms on this\ndata set. A multimodal system, on the other hand, is still able to provide high\naccuracy. Finally, the proposed cross fusion FCN was evaluated on the KITTI\nroad benchmark where it achieved excellent performance, with a MaxF score of\n96.03%, ranking it among the top-performing approaches.","url_abs":"http://arxiv.org/abs/1809.07941v1","url_pdf":"http://arxiv.org/pdf/1809.07941v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lidar-camera-fusion-for-road-detection-using","repo_url":"https://github.com/Shuijing725/ece498sm_project","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"fcn","method_name":"FCN"},{"method_slug":"max-pooling","method_name":"Max Pooling"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1809.07941","atlas_url":"https://app.syntology.ai/?focus=1809.07941","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}