{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/vortx-volumetric-3d-reconstruction-with","title":"VoRTX: Volumetric 3D Reconstruction With Transformers for Voxelwise View Selection and Fusion","arxiv_id":"2112.00236","date":"2021-12-01","proceeding":null,"authors":["Noah Stier","Alexander Rich","Pradeep Sen","Tobias Höllerer"],"abstract":"Recent volumetric 3D reconstruction methods can produce very accurate results, with plausible geometry even for unobserved surfaces. However, they face an undesirable trade-off when it comes to multi-view fusion. They can fuse all available view information by global averaging, thus losing fine detail, or they can heuristically cluster views for local fusion, thus restricting their ability to consider all views jointly. Our key insight is that greater detail can be retained without restricting view diversity by learning a view-fusion function conditioned on camera pose and image content. We propose to learn this multi-view fusion using a transformer. To this end, we introduce VoRTX, an end-to-end volumetric 3D reconstruction network using transformers for wide-baseline, multi-view feature fusion. Our model is occlusion-aware, leveraging the transformer architecture to predict an initial, projective scene geometry estimate. This estimate is used to avoid backprojecting image features through surfaces into occluded regions. We train our model on ScanNet and show that it produces better reconstructions than state-of-the-art methods. We also demonstrate generalization without any fine-tuning, outperforming the same state-of-the-art methods on two other datasets, TUM-RGBD and ICL-NUIM.","url_abs":"https://arxiv.org/abs/2112.00236v1","url_pdf":"https://arxiv.org/pdf/2112.00236v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"vortx-volumetric-3d-reconstruction-with","repo_url":"https://github.com/noahstier/vortx","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"},{"task_slug":"diversity","task_name":"Diversity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2112.00236","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.00236"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/noahstier/vortx","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"unverified":11},"by_repo_kind":{"official":{"samples":13,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"cb35e06cffb77cfc","entry":"get_view_frustum","repo":"noahstier/vortx","repo_kind":"official","path":"vortx/tsdf_fusion.py","file_url":"https://github.com/noahstier/vortx/blob/HEAD/vortx/tsdf_fusion.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb35e06cffb77cfc"}},{"code_sha256_prefix":"dc7f3e9cf45d3d8b","entry":"rigid_transform","repo":"noahstier/vortx","repo_kind":"official","path":"vortx/tsdf_fusion.py","file_url":"https://github.com/noahstier/vortx/blob/HEAD/vortx/tsdf_fusion.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dc7f3e9cf45d3d8b"}},{"code_sha256_prefix":"991ca46db203e09b","entry":"compute_proj_occ_loss","repo":"noahstier/vortx","repo_kind":"official","path":"vortx/vortx.py","file_url":"https://github.com/noahstier/vortx/blob/HEAD/vortx/vortx.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"991ca46db203e09b"}},{"code_sha256_prefix":"c9115ad6be42756f","entry":"eval_tsdf","repo":"noahstier/vortx","repo_kind":"official","path":"vortx/eval_utils.py","file_url":"https://github.com/noahstier/vortx/blob/HEAD/vortx/eval_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c9115ad6be42756f"}},{"code_sha256_prefix":"7c932be371f01e04","entry":"get_proj_mats","repo":"noahstier/vortx","repo_kind":"official","path":"vortx/data.py","file_url":"https://github.com/noahstier/vortx/blob/HEAD/vortx/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7c932be371f01e04"}},{"code_sha256_prefix":"a16f812079cf2701","entry":"get_subsample_inds","repo":"noahstier/vortx","repo_kind":"official","path":"vortx/vortx.py","file_url":"https://github.com/noahstier/vortx/blob/HEAD/vortx/vortx.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a16f812079cf2701"}},{"code_sha256_prefix":"fecd5f220c505215","entry":"integrate","repo":"noahstier/vortx","repo_kind":"official","path":"vortx/tsdf_fusion.py","file_url":"https://github.com/noahstier/vortx/blob/HEAD/vortx/tsdf_fusion.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fecd5f220c505215"}},{"code_sha256_prefix":"4983f48e58881a90","entry":"l1","repo":"noahstier/vortx","repo_kind":"official","path":"vortx/eval_utils.py","file_url":"https://github.com/noahstier/vortx/blob/HEAD/vortx/eval_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4983f48e58881a90"}},{"code_sha256_prefix":"656e784f281c3c4d","entry":"load_tsdf","repo":"noahstier/vortx","repo_kind":"official","path":"vortx/data.py","file_url":"https://github.com/noahstier/vortx/blob/HEAD/vortx/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"656e784f281c3c4d"}},{"code_sha256_prefix":"067cd91ea5bf39ff","entry":"log_transform","repo":"noahstier/vortx","repo_kind":"official","path":"vortx/utils.py","file_url":"https://github.com/noahstier/vortx/blob/HEAD/vortx/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"067cd91ea5bf39ff"}},{"code_sha256_prefix":"8bc9bfbe1fd3520d","entry":"mv_fusion_mean","repo":"noahstier/vortx","repo_kind":"official","path":"vortx/mv_fusion.py","file_url":"https://github.com/noahstier/vortx/blob/HEAD/vortx/mv_fusion.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8bc9bfbe1fd3520d"}},{"code_sha256_prefix":"789aa06f8613848a","entry":"reflect_pose","repo":"noahstier/vortx","repo_kind":"official","path":"vortx/data.py","file_url":"https://github.com/noahstier/vortx/blob/HEAD/vortx/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"789aa06f8613848a"}},{"code_sha256_prefix":"d59ce3497ebd1985","entry":"to_vol","repo":"noahstier/vortx","repo_kind":"official","path":"vortx/utils.py","file_url":"https://github.com/noahstier/vortx/blob/HEAD/vortx/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d59ce3497ebd1985"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}