{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sadg-segment-any-dynamic-gaussian-without","title":"SADG: Segment Any Dynamic Gaussian Without Object Trackers","arxiv_id":"2411.19290","date":"2024-11-28","proceeding":null,"authors":["Yun-Jin Li","Mariia Gladkova","Yan Xia","Daniel Cremers"],"abstract":"Understanding dynamic 3D scenes is fundamental for various applications, including extended reality (XR) and autonomous driving. Effectively integrating semantic information into 3D reconstruction enables holistic representation that opens opportunities for immersive and interactive applications. We introduce SADG, Segment Any Dynamic Gaussian Without Object Trackers, a novel approach that combines dynamic Gaussian Splatting representation and semantic information without reliance on object IDs. In contrast to existing works, we do not rely on supervision based on object identities to enable consistent segmentation of dynamic 3D objects. To this end, we propose to learn semantically-aware features by leveraging masks generated from the Segment Anything Model (SAM) and utilizing our novel contrastive learning objective based on hard pixel mining. The learned Gaussian features can be effectively clustered without further post-processing. This enables fast computation for further object-level editing, such as object removal, composition, and style transfer by manipulating the Gaussians in the scene. We further extend several dynamic novel-view datasets with segmentation benchmarks to enable testing of learned feature fields from unseen viewpoints. We evaluate SADG on proposed benchmarks and demonstrate the superior performance of our approach in segmenting objects within dynamic scenes along with its effectiveness for further downstream editing tasks.","url_abs":"https://arxiv.org/abs/2411.19290v1","url_pdf":"https://arxiv.org/pdf/2411.19290v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sadg-segment-any-dynamic-gaussian-without","repo_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"},{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"},{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"object","task_name":"Object"},{"task_slug":"style-transfer","task_name":"Style Transfer"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2411.19290","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.19290"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":1,"ran_draft_wrong":1,"unverified":15},"by_repo_kind":{"official":{"samples":17,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"75b25cc18e1773a8","entry":"closest_point_2_lines","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"neu3d2blender.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/neu3d2blender.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"75b25cc18e1773a8"}},{"code_sha256_prefix":"8d68829e9ebe344d","entry":"get_embedder","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"gui_standalone.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/gui_standalone.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8d68829e9ebe344d"}},{"code_sha256_prefix":"6cca3fa0d3e48ce1","entry":"array_to_blob","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"immersive2blender.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/immersive2blender.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6cca3fa0d3e48ce1"}},{"code_sha256_prefix":"6632742f237b890c","entry":"blob_to_array","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"immersive2blender.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/immersive2blender.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6632742f237b890c"}},{"code_sha256_prefix":"86791b215b923f7f","entry":"compute_acc","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"metrics_segmentation.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/metrics_segmentation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"86791b215b923f7f"}},{"code_sha256_prefix":"c68803bf7444deb3","entry":"compute_iou","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"metrics_segmentation.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/metrics_segmentation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c68803bf7444deb3"}},{"code_sha256_prefix":"e35aafbf0069edb6","entry":"dot","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"cam_utils.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/cam_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e35aafbf0069edb6"}},{"code_sha256_prefix":"2703b034706f4a1f","entry":"feature3d_to_rgb","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"gui.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/gui.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2703b034706f4a1f"}},{"code_sha256_prefix":"949d2f41c34cb63f","entry":"generate_grid_index","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"render.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/render.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"949d2f41c34cb63f"}},{"code_sha256_prefix":"a148f7f80321414f","entry":"getProjectionMatrix","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"gui.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/gui.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a148f7f80321414f"}},{"code_sha256_prefix":"919b604b3c2ecfbc","entry":"landmark_interpolate","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"gui.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/gui.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"919b604b3c2ecfbc"}},{"code_sha256_prefix":"43cdc168f88caf9b","entry":"length","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"cam_utils.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/cam_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43cdc168f88caf9b"}},{"code_sha256_prefix":"0b1363492bbc4c3c","entry":"multithread_write","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"render.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/render.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b1363492bbc4c3c"}},{"code_sha256_prefix":"74c6a993a0948239","entry":"prepare_output_and_logger","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"train_style_transfer_nnfm.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/train_style_transfer_nnfm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"74c6a993a0948239"}},{"code_sha256_prefix":"c97c897ae0d4ea0c","entry":"readImages","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"metrics_segmentation.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/metrics_segmentation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c97c897ae0d4ea0c"}},{"code_sha256_prefix":"1f4ce1f84eb70218","entry":"rotmat","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"immersive2blender.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/immersive2blender.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1f4ce1f84eb70218"}},{"code_sha256_prefix":"8a8484ccf552d87f","entry":"safe_normalize","repo":"yunjinli/SADG-SegmentAnyDynamicGaussian","repo_kind":"official","path":"cam_utils.py","file_url":"https://github.com/yunjinli/SADG-SegmentAnyDynamicGaussian/blob/HEAD/cam_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8a8484ccf552d87f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}