{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/3dgut-enabling-distorted-cameras-and","title":"3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting","arxiv_id":"2412.12507","date":"2024-12-17","proceeding":"CVPR 2025 1","authors":["Qi Wu","Janick Martinez Esturo","Ashkan Mirzaei","Nicolas Moenne-Loccoz","Zan Gojcic"],"abstract":"3D Gaussian Splatting (3DGS) enables efficient reconstruction and high-fidelity real-time rendering of complex scenes on consumer hardware. However, due to its rasterization-based formulation, 3DGS is constrained to ideal pinhole cameras and lacks support for secondary lighting effects. Recent methods address these limitations by tracing the particles instead, but, this comes at the cost of significantly slower rendering. In this work, we propose 3D Gaussian Unscented Transform (3DGUT), replacing the EWA splatting formulation with the Unscented Transform that approximates the particles through sigma points, which can be projected exactly under any nonlinear projection function. This modification enables trivial support of distorted cameras with time dependent effects such as rolling shutter, while retaining the efficiency of rasterization. Additionally, we align our rendering formulation with that of tracing-based methods, enabling secondary ray tracing required to represent phenomena such as reflections and refraction within the same 3D representation. The source code is available at: https://github.com/nv-tlabs/3dgrut.","url_abs":"https://arxiv.org/abs/2412.12507v2","url_pdf":"https://arxiv.org/pdf/2412.12507v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"3dgut-enabling-distorted-cameras-and","repo_url":"https://github.com/nv-tlabs/3dgrut","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3dgs","task_name":"3DGS"},{"task_slug":"novel-view-synthesis","task_name":"Novel View Synthesis"}],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/novel-view-synthesis-on-scannet","task":"Novel View Synthesis","dataset":"ScanNet++","model":"3DGUT","rank_in_archive_order":2,"of":4,"metrics":{"LPIPS":"0.150","PSNR":"30.64","SSIM":"0.944"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2412.12507","atlas_url":"https://app.syntology.ai/?focus=2412.12507","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.12507"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nv-tlabs/3dgrut","reach":null}],"summary":{"ran_violates":1,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"96334775f95b474e","entry":"compute_runtime_post_processing","repo":"nv-tlabs/3dgrut","repo_kind":"official","path":"threedgrut/export/usd/exporter.py","file_url":"https://github.com/nv-tlabs/3dgrut/blob/HEAD/threedgrut/export/usd/exporter.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"96334775f95b474e"}},{"code_sha256_prefix":"b082f4bc8cc062e6","entry":"normalize_ppisp_integration_mode","repo":"nv-tlabs/3dgrut","repo_kind":"official","path":"threedgrut/export/usd/exporter.py","file_url":"https://github.com/nv-tlabs/3dgrut/blob/HEAD/threedgrut/export/usd/exporter.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b082f4bc8cc062e6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}