{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bootstraping-clustering-of-gaussians-for-view","title":"Bootstraping Clustering of Gaussians for View-consistent 3D Scene Understanding","arxiv_id":"2411.19551","date":"2024-11-29","proceeding":null,"authors":["Wenbo Zhang","Lu Zhang","Ping Hu","Liqian Ma","Yunzhi Zhuge","Huchuan Lu"],"abstract":"Injecting semantics into 3D Gaussian Splatting (3DGS) has recently garnered significant attention. While current approaches typically distill 3D semantic features from 2D foundational models (e.g., CLIP and SAM) to facilitate novel view segmentation and semantic understanding, their heavy reliance on 2D supervision can undermine cross-view semantic consistency and necessitate complex data preparation processes, therefore hindering view-consistent scene understanding. In this work, we present FreeGS, an unsupervised semantic-embedded 3DGS framework that achieves view-consistent 3D scene understanding without the need for 2D labels. Instead of directly learning semantic features, we introduce the IDentity-coupled Semantic Field (IDSF) into 3DGS, which captures both semantic representations and view-consistent instance indices for each Gaussian. We optimize IDSF with a two-step alternating strategy: semantics help to extract coherent instances in 3D space, while the resulting instances regularize the injection of stable semantics from 2D space. Additionally, we adopt a 2D-3D joint contrastive loss to enhance the complementarity between view-consistent 3D geometry and rich semantics during the bootstrapping process, enabling FreeGS to uniformly perform tasks such as novel-view semantic segmentation, object selection, and 3D object detection. Extensive experiments on LERF-Mask, 3D-OVS, and ScanNet datasets demonstrate that FreeGS performs comparably to state-of-the-art methods while avoiding the complex data preprocessing workload. Our code is publicly available at https://github.com/wb014/FreeGS.","url_abs":"https://arxiv.org/abs/2411.19551v2","url_pdf":"https://arxiv.org/pdf/2411.19551v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bootstraping-clustering-of-gaussians-for-view","repo_url":"https://github.com/wb014/FreeGS","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"3d-object-detection","task_name":"3D Object Detection"},{"task_slug":"3d-geometry","task_name":"3D geometry"},{"task_slug":"3dgs","task_name":"3DGS"},{"task_slug":"clustering","task_name":"Clustering"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"adopt","method_name":"ADOPT"},{"method_slug":"clip","method_name":"CLIP"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2411.19551","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.19551"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wb014/FreeGS","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"ran_honours":2,"ran":2,"unverified":7},"by_repo_kind":{"official":{"samples":12,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"7325f65ee1e29543","entry":"get_expon_lr_func","repo":"wb014/FreeGS","repo_kind":"official","path":"utils/general_utils.py","file_url":"https://github.com/wb014/FreeGS/blob/HEAD/utils/general_utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7325f65ee1e29543"}},{"code_sha256_prefix":"b488da571728b636","entry":"inverse_sigmoid","repo":"wb014/FreeGS","repo_kind":"official","path":"utils/general_utils.py","file_url":"https://github.com/wb014/FreeGS/blob/HEAD/utils/general_utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b488da571728b636"}},{"code_sha256_prefix":"ac0e42d6fbcfbbe6","entry":"l1_loss","repo":"wb014/FreeGS","repo_kind":"official","path":"utils/loss_utils.py","file_url":"https://github.com/wb014/FreeGS/blob/HEAD/utils/loss_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ac0e42d6fbcfbbe6"}},{"code_sha256_prefix":"8c3b0f873ba11813","entry":"l2_loss","repo":"wb014/FreeGS","repo_kind":"official","path":"utils/loss_utils.py","file_url":"https://github.com/wb014/FreeGS/blob/HEAD/utils/loss_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8c3b0f873ba11813"}},{"code_sha256_prefix":"56858e04e6fdb2ff","entry":"read_next_bytes","repo":"wb014/FreeGS","repo_kind":"official","path":"scene/colmap_loader.py","file_url":"https://github.com/wb014/FreeGS/blob/HEAD/scene/colmap_loader.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"56858e04e6fdb2ff"}},{"code_sha256_prefix":"301bd8c9fc8769e3","entry":"PILtoTorch","repo":"wb014/FreeGS","repo_kind":"official","path":"utils/general_utils.py","file_url":"https://github.com/wb014/FreeGS/blob/HEAD/utils/general_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"301bd8c9fc8769e3"}},{"code_sha256_prefix":"ef11a6b3782463a1","entry":"dilate","repo":"wb014/FreeGS","repo_kind":"official","path":"train_freegs.py","file_url":"https://github.com/wb014/FreeGS/blob/HEAD/train_freegs.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ef11a6b3782463a1"}},{"code_sha256_prefix":"1420b25f81e3e725","entry":"distCUDA2","repo":"wb014/FreeGS","repo_kind":"official","path":"scene/gaussian_model.py","file_url":"https://github.com/wb014/FreeGS/blob/HEAD/scene/gaussian_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1420b25f81e3e725"}},{"code_sha256_prefix":"2ffdf340fdd5e487","entry":"generate_unique_colors","repo":"wb014/FreeGS","repo_kind":"official","path":"train_freegs.py","file_url":"https://github.com/wb014/FreeGS/blob/HEAD/train_freegs.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2ffdf340fdd5e487"}},{"code_sha256_prefix":"28862fced4963b57","entry":"ins_l1_loss","repo":"wb014/FreeGS","repo_kind":"official","path":"utils/loss_utils.py","file_url":"https://github.com/wb014/FreeGS/blob/HEAD/utils/loss_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"28862fced4963b57"}},{"code_sha256_prefix":"6ce64cf0fbcd6bb1","entry":"qvec2rotmat","repo":"wb014/FreeGS","repo_kind":"official","path":"scene/colmap_loader.py","file_url":"https://github.com/wb014/FreeGS/blob/HEAD/scene/colmap_loader.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6ce64cf0fbcd6bb1"}},{"code_sha256_prefix":"659bc4e7e63ed8f9","entry":"rotmat2qvec","repo":"wb014/FreeGS","repo_kind":"official","path":"scene/colmap_loader.py","file_url":"https://github.com/wb014/FreeGS/blob/HEAD/scene/colmap_loader.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"659bc4e7e63ed8f9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}