{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bundlesdf-neural-6-dof-tracking-and-3d","title":"BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects","arxiv_id":"2303.14158","date":"2023-03-24","proceeding":"CVPR 2023 1","authors":["Bowen Wen","Jonathan Tremblay","Valts Blukis","Stephen Tyree","Thomas Muller","Alex Evans","Dieter Fox","Jan Kautz","Stan Birchfield"],"abstract":"We present a near real-time method for 6-DoF tracking of an unknown object from a monocular RGBD video sequence, while simultaneously performing neural 3D reconstruction of the object. Our method works for arbitrary rigid objects, even when visual texture is largely absent. The object is assumed to be segmented in the first frame only. No additional information is required, and no assumption is made about the interaction agent. Key to our method is a Neural Object Field that is learned concurrently with a pose graph optimization process in order to robustly accumulate information into a consistent 3D representation capturing both geometry and appearance. A dynamic pool of posed memory frames is automatically maintained to facilitate communication between these threads. Our approach handles challenging sequences with large pose changes, partial and full occlusion, untextured surfaces, and specular highlights. We show results on HO3D, YCBInEOAT, and BEHAVE datasets, demonstrating that our method significantly outperforms existing approaches. Project page: https://bundlesdf.github.io","url_abs":"https://arxiv.org/abs/2303.14158v1","url_pdf":"https://arxiv.org/pdf/2303.14158v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bundlesdf-neural-6-dof-tracking-and-3d","repo_url":"https://github.com/NVlabs/BundleSDF","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"3d-object-detection","task_name":"3D Object Detection"},{"task_slug":"3d-object-tracking","task_name":"3D Object Tracking"},{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"},{"task_slug":"6d-pose-estimation-1","task_name":"6D Pose Estimation"},{"task_slug":"6d-pose-estimation-using-rgbd","task_name":"6D Pose Estimation using RGBD"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-tracking","task_name":"Object Tracking"},{"task_slug":"pose-tracking","task_name":"Pose Tracking"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-object-detection-on-dttd2","task":"3D Object Detection","dataset":"DTTD-Mobile","model":"BundleSDF","rank_in_archive_order":4,"of":5,"metrics":{"ADD AUC":"46.86","ADD-S AUC":"55.74"},"uses_additional_data":false},{"leaderboard":"/sota/6d-pose-estimation-on-dttd2","task":"6D Pose Estimation","dataset":"DTTD-Mobile","model":"BundleSDF","rank_in_archive_order":3,"of":8,"metrics":{"ADD AUC":"46.86","ADD-S AUC":"55.74"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2303.14158","atlas_url":"https://app.syntology.ai/?focus=2303.14158","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}