{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sparse-and-noisy-lidar-completion-with-rgb","title":"Sparse and noisy LiDAR completion with RGB guidance and uncertainty","arxiv_id":"1902.05356","date":"2019-02-14","proceeding":null,"authors":["Wouter Van Gansbeke","Davy Neven","Bert de Brabandere","Luc van Gool"],"abstract":"This work proposes a new method to accurately complete sparse LiDAR maps\nguided by RGB images. For autonomous vehicles and robotics the use of LiDAR is\nindispensable in order to achieve precise depth predictions. A multitude of\napplications depend on the awareness of their surroundings, and use depth cues\nto reason and react accordingly. On the one hand, monocular depth prediction\nmethods fail to generate absolute and precise depth maps. On the other hand,\nstereoscopic approaches are still significantly outperformed by LiDAR based\napproaches. The goal of the depth completion task is to generate dense depth\npredictions from sparse and irregular point clouds which are mapped to a 2D\nplane. We propose a new framework which extracts both global and local\ninformation in order to produce proper depth maps. We argue that simple depth\ncompletion does not require a deep network. However, we additionally propose a\nfusion method with RGB guidance from a monocular camera in order to leverage\nobject information and to correct mistakes in the sparse input. This improves\nthe accuracy significantly. Moreover, confidence masks are exploited in order\nto take into account the uncertainty in the depth predictions from each\nmodality. This fusion method outperforms the state-of-the-art and ranks first\non the KITTI depth completion benchmark. Our code with visualizations is\navailable.","url_abs":"http://arxiv.org/abs/1902.05356v1","url_pdf":"http://arxiv.org/pdf/1902.05356v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sparse-and-noisy-lidar-completion-with-rgb","repo_url":"https://github.com/wvangansbeke/Sparse-Depth-Completion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"autonomous-vehicles","task_name":"Autonomous Vehicles"},{"task_slug":"depth-completion","task_name":"Depth Completion"},{"task_slug":"depth-estimation","task_name":"Depth Estimation"},{"task_slug":"depth-prediction","task_name":"Depth Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/depth-completion-on-kitti-depth-completion","task":"Depth Completion","dataset":"KITTI Depth Completion","model":"FusionNet (RGB_guide&certainty)","rank_in_archive_order":5,"of":16,"metrics":{"MAE":"215.02","RMSE":"772.87","Runtime [ms]":"20","iMAE":"0.93","iRMSE":"2.19"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1902.05356","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}