{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/feature-3dgs-supercharging-3d-gaussian","title":"Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields","arxiv_id":"2312.03203","date":"2023-12-06","proceeding":"CVPR 2024 1","authors":["Shijie Zhou","Haoran Chang","Sicheng Jiang","Zhiwen Fan","Zehao Zhu","Dejia Xu","Pradyumna Chari","Suya You","Zhangyang Wang","Achuta Kadambi"],"abstract":"3D scene representations have gained immense popularity in recent years. Methods that use Neural Radiance fields are versatile for traditional tasks such as novel view synthesis. In recent times, some work has emerged that aims to extend the functionality of NeRF beyond view synthesis, for semantically aware tasks such as editing and segmentation using 3D feature field distillation from 2D foundation models. However, these methods have two major limitations: (a) they are limited by the rendering speed of NeRF pipelines, and (b) implicitly represented feature fields suffer from continuity artifacts reducing feature quality. Recently, 3D Gaussian Splatting has shown state-of-the-art performance on real-time radiance field rendering. In this work, we go one step further: in addition to radiance field rendering, we enable 3D Gaussian splatting on arbitrary-dimension semantic features via 2D foundation model distillation. This translation is not straightforward: naively incorporating feature fields in the 3DGS framework encounters significant challenges, notably the disparities in spatial resolution and channel consistency between RGB images and feature maps. We propose architectural and training changes to efficiently avert this problem. Our proposed method is general, and our experiments showcase novel view semantic segmentation, language-guided editing and segment anything through learning feature fields from state-of-the-art 2D foundation models such as SAM and CLIP-LSeg. Across experiments, our distillation method is able to provide comparable or better results, while being significantly faster to both train and render. Additionally, to the best of our knowledge, we are the first method to enable point and bounding-box prompting for radiance field manipulation, by leveraging the SAM model. Project website at: https://feature-3dgs.github.io/","url_abs":"https://arxiv.org/abs/2312.03203v3","url_pdf":"https://arxiv.org/pdf/2312.03203v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"feature-3dgs-supercharging-3d-gaussian","repo_url":"https://github.com/ShijieZhou-UCLA/feature-3dgs","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"feature-3dgs-supercharging-3d-gaussian","repo_url":"https://github.com/keloee/maskfield","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-scene-editing","task_name":"3D scene Editing"},{"task_slug":"3dgs","task_name":"3DGS"},{"task_slug":"natural-language-queries","task_name":"Natural Language Queries"},{"task_slug":"nerf","task_name":"NeRF"},{"task_slug":"novel-view-synthesis","task_name":"Novel View Synthesis"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"aware","method_name":"AWARE"},{"method_slug":"sam","method_name":"SAM"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.03203","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.03203"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ShijieZhou-UCLA/feature-3dgs","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/keloee/maskfield","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":7,"ran_fixture":3,"ran_violates":1,"ran_draft_wrong":1,"unverified":12},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1},"listed":{"samples":23,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"eae018b803b70a00","entry":"OctreeRender_trilinear_fast_depth","repo":"keloee/maskfield","repo_kind":"listed","path":"renderer.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/renderer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eae018b803b70a00"}},{"code_sha256_prefix":"b73525bcfb4b4e5a","entry":"batch_dice_loss","repo":"keloee/maskfield","repo_kind":"listed","path":"models/matcher.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/models/matcher.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b73525bcfb4b4e5a"}},{"code_sha256_prefix":"016dba4163da28df","entry":"dice_loss","repo":"keloee/maskfield","repo_kind":"listed","path":"models/losses.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/models/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"016dba4163da28df"}},{"code_sha256_prefix":"db95ecdf5d4abe6b","entry":"eval_sh","repo":"keloee/maskfield","repo_kind":"listed","path":"models/sh.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/models/sh.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"db95ecdf5d4abe6b"}},{"code_sha256_prefix":"caa9e209a9083a0f","entry":"eval_sh_bases","repo":"keloee/maskfield","repo_kind":"listed","path":"models/sh.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/models/sh.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"caa9e209a9083a0f"}},{"code_sha256_prefix":"2e7f3dd0cf98f11e","entry":"filter","repo":"keloee/maskfield","repo_kind":"listed","path":"extract_feature.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/extract_feature.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2e7f3dd0cf98f11e"}},{"code_sha256_prefix":"826d97f0a222f986","entry":"get_legend_patch","repo":"ShijieZhou-UCLA/feature-3dgs","repo_kind":"official","path":"encoders/lseg_encoder/encode_images.py","file_url":"https://github.com/ShijieZhou-UCLA/feature-3dgs/blob/HEAD/encoders/lseg_encoder/encode_images.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"826d97f0a222f986"}},{"code_sha256_prefix":"dc510fe4f2261490","entry":"init_log","repo":"keloee/maskfield","repo_kind":"listed","path":"funcs.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/funcs.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dc510fe4f2261490"}},{"code_sha256_prefix":"f062653540840f44","entry":"pad_img","repo":"keloee/maskfield","repo_kind":"listed","path":"extract_feature.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/extract_feature.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f062653540840f44"}},{"code_sha256_prefix":"11a10efa60767162","entry":"positional_encoding","repo":"keloee/maskfield","repo_kind":"listed","path":"models/tensorBase.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/models/tensorBase.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"11a10efa60767162"}},{"code_sha256_prefix":"c1acffa981204fee","entry":"raw2alpha","repo":"keloee/maskfield","repo_kind":"listed","path":"models/tensorBase.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/models/tensorBase.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c1acffa981204fee"}},{"code_sha256_prefix":"c775698c33cfa2de","entry":"visualize_depth_numpy","repo":"keloee/maskfield","repo_kind":"listed","path":"funcs.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/funcs.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c775698c33cfa2de"}},{"code_sha256_prefix":"6a37a8e809bde7a0","entry":"OctreeRender_trilinear_fast","repo":"keloee/maskfield","repo_kind":"listed","path":"renderer.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/renderer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a37a8e809bde7a0"}},{"code_sha256_prefix":"3a530f556e40d9c0","entry":"SHRender","repo":"keloee/maskfield","repo_kind":"listed","path":"models/tensorBase.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/models/tensorBase.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a530f556e40d9c0"}},{"code_sha256_prefix":"72f9a57ae3d0ae8d","entry":"apply_float_colormap","repo":"keloee/maskfield","repo_kind":"listed","path":"colormaps.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/colormaps.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"72f9a57ae3d0ae8d"}},{"code_sha256_prefix":"5ce0a97aafc3b0d4","entry":"batch_sigmoid_focal_loss","repo":"keloee/maskfield","repo_kind":"listed","path":"models/matcher.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/models/matcher.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ce0a97aafc3b0d4"}},{"code_sha256_prefix":"88f81a32c328085d","entry":"chunk_cosine_sim","repo":"keloee/maskfield","repo_kind":"listed","path":"models/matcher.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/models/matcher.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"88f81a32c328085d"}},{"code_sha256_prefix":"0b4c49066bad50f8","entry":"evaluation","repo":"keloee/maskfield","repo_kind":"listed","path":"renderer.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/renderer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b4c49066bad50f8"}},{"code_sha256_prefix":"eeee61bc904ad6c5","entry":"get_color","repo":"keloee/maskfield","repo_kind":"listed","path":"colors.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/colors.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eeee61bc904ad6c5"}},{"code_sha256_prefix":"8814888ea31454d6","entry":"js_divergence","repo":"keloee/maskfield","repo_kind":"listed","path":"models/losses.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/models/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8814888ea31454d6"}},{"code_sha256_prefix":"d4b16e8ecb449d48","entry":"mask_nms","repo":"keloee/maskfield","repo_kind":"listed","path":"extract_feature.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/extract_feature.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d4b16e8ecb449d48"}},{"code_sha256_prefix":"2da4941aadc1be82","entry":"py_sigmoid_focal_loss","repo":"keloee/maskfield","repo_kind":"listed","path":"models/focal_loss.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/models/focal_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2da4941aadc1be82"}},{"code_sha256_prefix":"71e149dc0c33b040","entry":"sigmoid_focal_loss","repo":"keloee/maskfield","repo_kind":"listed","path":"models/losses.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/models/losses.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"71e149dc0c33b040"}},{"code_sha256_prefix":"0b78c3cf1e82d884","entry":"weight_reduce_loss","repo":"keloee/maskfield","repo_kind":"listed","path":"models/focal_loss.py","file_url":"https://github.com/keloee/maskfield/blob/HEAD/models/focal_loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b78c3cf1e82d884"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}