{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/clip-nerf-text-and-image-driven-manipulation","title":"CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields","arxiv_id":"2112.05139","date":"2021-12-09","proceeding":"CVPR 2022 1","authors":["Can Wang","Menglei Chai","Mingming He","Dongdong Chen","Jing Liao"],"abstract":"We present CLIP-NeRF, a multi-modal 3D object manipulation method for neural radiance fields (NeRF). By leveraging the joint language-image embedding space of the recent Contrastive Language-Image Pre-Training (CLIP) model, we propose a unified framework that allows manipulating NeRF in a user-friendly way, using either a short text prompt or an exemplar image. Specifically, to combine the novel view synthesis capability of NeRF and the controllable manipulation ability of latent representations from generative models, we introduce a disentangled conditional NeRF architecture that allows individual control over both shape and appearance. This is achieved by performing the shape conditioning via applying a learned deformation field to the positional encoding and deferring color conditioning to the volumetric rendering stage. To bridge this disentangled latent representation to the CLIP embedding, we design two code mappers that take a CLIP embedding as input and update the latent codes to reflect the targeted editing. The mappers are trained with a CLIP-based matching loss to ensure the manipulation accuracy. Furthermore, we propose an inverse optimization method that accurately projects an input image to the latent codes for manipulation to enable editing on real images. We evaluate our approach by extensive experiments on a variety of text prompts and exemplar images and also provide an intuitive interface for interactive editing. Our implementation is available at https://cassiepython.github.io/clipnerf/","url_abs":"https://arxiv.org/abs/2112.05139v3","url_pdf":"https://arxiv.org/pdf/2112.05139v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"clip-nerf-text-and-image-driven-manipulation","repo_url":"https://github.com/cassiePython/CLIPNeRF","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"clip-nerf-text-and-image-driven-manipulation","repo_url":"https://github.com/ashawkey/torch-ngp","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"nerf","task_name":"NeRF"},{"task_slug":"novel-view-synthesis","task_name":"Novel View Synthesis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2112.05139","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.05139"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cassiePython/CLIPNeRF","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ashawkey/torch-ngp","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d2db52c47d9a9deb","entry":"batchify","repo":"cassiePython/CLIPNeRF","repo_kind":"official","path":"supplyment/run_nerf_clip.py","file_url":"https://github.com/cassiePython/CLIPNeRF/blob/HEAD/supplyment/run_nerf_clip.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d2db52c47d9a9deb"}},{"code_sha256_prefix":"2fba756fe9b544aa","entry":"get_select_inds","repo":"cassiePython/CLIPNeRF","repo_kind":"official","path":"supplyment/run_nerf_clip.py","file_url":"https://github.com/cassiePython/CLIPNeRF/blob/HEAD/supplyment/run_nerf_clip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2fba756fe9b544aa"}},{"code_sha256_prefix":"4042ffc9f214a037","entry":"run_network","repo":"cassiePython/CLIPNeRF","repo_kind":"official","path":"supplyment/run_nerf_clip.py","file_url":"https://github.com/cassiePython/CLIPNeRF/blob/HEAD/supplyment/run_nerf_clip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4042ffc9f214a037"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}