{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/monocular-scene-reconstruction-with-3d-sdf","title":"3D Former: Monocular Scene Reconstruction with 3D SDF Transformers","arxiv_id":"2301.13510","date":"2023-01-31","proceeding":null,"authors":["Weihao Yuan","Xiaodong Gu","Heng Li","Zilong Dong","Siyu Zhu"],"abstract":"Monocular scene reconstruction from posed images is challenging due to the complexity of a large environment. Recent volumetric methods learn to directly predict the TSDF volume and have demonstrated promising results in this task. However, most methods focus on how to extract and fuse the 2D features to a 3D feature volume, but none of them improve the way how the 3D volume is aggregated. In this work, we propose an SDF transformer network, which replaces the role of 3D CNN for better 3D feature aggregation. To reduce the explosive computation complexity of the 3D multi-head attention, we propose a sparse window attention module, where the attention is only calculated between the non-empty voxels within a local window. Then a top-down-bottom-up 3D attention network is built for 3D feature aggregation, where a dilate-attention structure is proposed to prevent geometry degeneration, and two global modules are employed to equip with global receptive fields. The experiments on multiple datasets show that this 3D transformer network generates a more accurate and complete reconstruction, which outperforms previous methods by a large margin. Remarkably, the mesh accuracy is improved by 41.8%, and the mesh completeness is improved by 25.3% on the ScanNet dataset. Project page: https://weihaosky.github.io/sdfformer.","url_abs":"https://arxiv.org/abs/2301.13510v2","url_pdf":"https://arxiv.org/pdf/2301.13510v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"monocular-scene-reconstruction-with-3d-sdf","repo_url":"https://github.com/alibaba-damo-academy/former3d","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"3d-cnn","method_name":"3D CNN"},{"method_slug":null,"method_name":"None"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2301.13510","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2301.13510"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alibaba-damo-academy/former3d","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"unverified":12},"by_repo_kind":{"official":{"samples":14,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"cb35e06cffb77cfc","entry":"get_view_frustum","repo":"alibaba-damo-academy/former3d","repo_kind":"official","path":"former3d/tsdf_fusion.py","file_url":"https://github.com/alibaba-damo-academy/former3d/blob/HEAD/former3d/tsdf_fusion.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"cb35e06cffb77cfc"}},{"code_sha256_prefix":"dc7f3e9cf45d3d8b","entry":"rigid_transform","repo":"alibaba-damo-academy/former3d","repo_kind":"official","path":"former3d/tsdf_fusion.py","file_url":"https://github.com/alibaba-damo-academy/former3d/blob/HEAD/former3d/tsdf_fusion.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"dc7f3e9cf45d3d8b"}},{"code_sha256_prefix":"991ca46db203e09b","entry":"compute_proj_occ_loss","repo":"alibaba-damo-academy/former3d","repo_kind":"official","path":"former3d/sdfformer.py","file_url":"https://github.com/alibaba-damo-academy/former3d/blob/HEAD/former3d/sdfformer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"991ca46db203e09b"}},{"code_sha256_prefix":"7c932be371f01e04","entry":"get_proj_mats","repo":"alibaba-damo-academy/former3d","repo_kind":"official","path":"former3d/data.py","file_url":"https://github.com/alibaba-damo-academy/former3d/blob/HEAD/former3d/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"7c932be371f01e04"}},{"code_sha256_prefix":"a16f812079cf2701","entry":"get_subsample_inds","repo":"alibaba-damo-academy/former3d","repo_kind":"official","path":"former3d/sdfformer.py","file_url":"https://github.com/alibaba-damo-academy/former3d/blob/HEAD/former3d/sdfformer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"a16f812079cf2701"}},{"code_sha256_prefix":"fecd5f220c505215","entry":"integrate","repo":"alibaba-damo-academy/former3d","repo_kind":"official","path":"former3d/tsdf_fusion.py","file_url":"https://github.com/alibaba-damo-academy/former3d/blob/HEAD/former3d/tsdf_fusion.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"fecd5f220c505215"}},{"code_sha256_prefix":"656e784f281c3c4d","entry":"load_tsdf","repo":"alibaba-damo-academy/former3d","repo_kind":"official","path":"former3d/data.py","file_url":"https://github.com/alibaba-damo-academy/former3d/blob/HEAD/former3d/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"656e784f281c3c4d"}},{"code_sha256_prefix":"067cd91ea5bf39ff","entry":"log_transform","repo":"alibaba-damo-academy/former3d","repo_kind":"official","path":"former3d/utils.py","file_url":"https://github.com/alibaba-damo-academy/former3d/blob/HEAD/former3d/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"067cd91ea5bf39ff"}},{"code_sha256_prefix":"c2b771a2c3dc45eb","entry":"mv_fusion_mean","repo":"alibaba-damo-academy/former3d","repo_kind":"official","path":"former3d/mv_fusion.py","file_url":"https://github.com/alibaba-damo-academy/former3d/blob/HEAD/former3d/mv_fusion.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"c2b771a2c3dc45eb"}},{"code_sha256_prefix":"789aa06f8613848a","entry":"reflect_pose","repo":"alibaba-damo-academy/former3d","repo_kind":"official","path":"former3d/data.py","file_url":"https://github.com/alibaba-damo-academy/former3d/blob/HEAD/former3d/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"789aa06f8613848a"}},{"code_sha256_prefix":"0737dd8ecef8e21b","entry":"scatter_nd","repo":"alibaba-damo-academy/former3d","repo_kind":"official","path":"former3d/net3d/sparse3d.py","file_url":"https://github.com/alibaba-damo-academy/former3d/blob/HEAD/former3d/net3d/sparse3d.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"0737dd8ecef8e21b"}},{"code_sha256_prefix":"47e615b4b491d0cb","entry":"sparse_collate_fn","repo":"alibaba-damo-academy/former3d","repo_kind":"official","path":"former3d/collate.py","file_url":"https://github.com/alibaba-damo-academy/former3d/blob/HEAD/former3d/collate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"47e615b4b491d0cb"}},{"code_sha256_prefix":"d44eba57e4c78fd4","entry":"spconvTensor2formerTensor","repo":"alibaba-damo-academy/former3d","repo_kind":"official","path":"former3d/net3d/sparse3d.py","file_url":"https://github.com/alibaba-damo-academy/former3d/blob/HEAD/former3d/net3d/sparse3d.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"d44eba57e4c78fd4"}},{"code_sha256_prefix":"d59ce3497ebd1985","entry":"to_vol","repo":"alibaba-damo-academy/former3d","repo_kind":"official","path":"former3d/utils.py","file_url":"https://github.com/alibaba-damo-academy/former3d/blob/HEAD/former3d/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"d59ce3497ebd1985"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}