{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bundlefusion-real-time-globally-consistent-3d","title":"BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration","arxiv_id":"1604.01093","date":"2016-04-05","proceeding":null,"authors":["Angela Dai","Matthias Nießner","Michael Zollhöfer","Shahram Izadi","Christian Theobalt"],"abstract":"Real-time, high-quality, 3D scanning of large-scale scenes is key to mixed\nreality and robotic applications. However, scalability brings challenges of\ndrift in pose estimation, introducing significant errors in the accumulated\nmodel. Approaches often require hours of offline processing to globally correct\nmodel errors. Recent online methods demonstrate compelling results, but suffer\nfrom: (1) needing minutes to perform online correction preventing true\nreal-time use; (2) brittle frame-to-frame (or frame-to-model) pose estimation\nresulting in many tracking failures; or (3) supporting only unstructured\npoint-based representations, which limit scan quality and applicability. We\nsystematically address these issues with a novel, real-time, end-to-end\nreconstruction framework. At its core is a robust pose estimation strategy,\noptimizing per frame for a global set of camera poses by considering the\ncomplete history of RGB-D input with an efficient hierarchical approach. We\nremove the heavy reliance on temporal tracking, and continually localize to the\nglobally optimized frames instead. We contribute a parallelizable optimization\nframework, which employs correspondences based on sparse features and dense\ngeometric and photometric matching. Our approach estimates globally optimized\n(i.e., bundle adjusted) poses in real-time, supports robust tracking with\nrecovery from gross tracking failures (i.e., relocalization), and re-estimates\nthe 3D model in real-time to ensure global consistency; all within a single\nframework. Our approach outperforms state-of-the-art online systems with\nquality on par to offline methods, but with unprecedented speed and scan\ncompleteness. Our framework leads to a comprehensive online scanning solution\nfor large indoor environments, enabling ease of use and high-quality results.","url_abs":"http://arxiv.org/abs/1604.01093v3","url_pdf":"http://arxiv.org/pdf/1604.01093v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bundlefusion-real-time-globally-consistent-3d","repo_url":"https://github.com/apple/ml-live-pose","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"},{"task_slug":"mixed-reality","task_name":"Mixed Reality"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1604.01093","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}