{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/prolificdreamer-high-fidelity-and-diverse","title":"ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation","arxiv_id":"2305.16213","date":"2023-05-25","proceeding":"NeurIPS 2023 11","authors":["Zhengyi Wang","Cheng Lu","Yikai Wang","Fan Bao","Chongxuan Li","Hang Su","Jun Zhu"],"abstract":"Score distillation sampling (SDS) has shown great promise in text-to-3D generation by distilling pretrained large-scale text-to-image diffusion models, but suffers from over-saturation, over-smoothing, and low-diversity problems. In this work, we propose to model the 3D parameter as a random variable instead of a constant as in SDS and present variational score distillation (VSD), a principled particle-based variational framework to explain and address the aforementioned issues in text-to-3D generation. We show that SDS is a special case of VSD and leads to poor samples with both small and large CFG weights. In comparison, VSD works well with various CFG weights as ancestral sampling from diffusion models and simultaneously improves the diversity and sample quality with a common CFG weight (i.e., $7.5$). We further present various improvements in the design space for text-to-3D such as distillation time schedule and density initialization, which are orthogonal to the distillation algorithm yet not well explored. Our overall approach, dubbed ProlificDreamer, can generate high rendering resolution (i.e., $512\\times512$) and high-fidelity NeRF with rich structure and complex effects (e.g., smoke and drops). Further, initialized from NeRF, meshes fine-tuned by VSD are meticulously detailed and photo-realistic. Project page and codes: https://ml.cs.tsinghua.edu.cn/prolificdreamer/","url_abs":"https://arxiv.org/abs/2305.16213v2","url_pdf":"https://arxiv.org/pdf/2305.16213v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"prolificdreamer-high-fidelity-and-diverse","repo_url":"https://github.com/threestudio-project/threestudio","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":null},{"paper_slug":"prolificdreamer-high-fidelity-and-diverse","repo_url":"https://github.com/yuanzhi-zhu/prolific_dreamer2d","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-generation","task_name":"3D Generation"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"nerf","task_name":"NeRF"},{"task_slug":"text-to-3d","task_name":"Text to 3D"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2305.16213","atlas_url":"https://app.syntology.ai/?focus=2305.16213","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.16213"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yuanzhi-zhu/prolific_dreamer2d","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/threestudio-project/threestudio","reach":null}],"summary":{"ran_draft_wrong":2,"unverified":3},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1},"listed":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"aba8623ee1020604","entry":"predict_noise0_diffuser","repo":"yuanzhi-zhu/prolific_dreamer2d","repo_kind":"listed","path":"model_utils.py","file_url":"https://github.com/yuanzhi-zhu/prolific_dreamer2d/blob/HEAD/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aba8623ee1020604"}},{"code_sha256_prefix":"e93da14f176f38af","entry":"tail","repo":"threestudio-project/threestudio","repo_kind":"official","path":"gradio_app.py","file_url":"https://github.com/threestudio-project/threestudio/blob/HEAD/gradio_app.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e93da14f176f38af"}},{"code_sha256_prefix":"849b90aa3c658242","entry":"on_model_selector_change","repo":"threestudio-project/threestudio","repo_kind":"official","path":"gradio_app.py","file_url":"https://github.com/threestudio-project/threestudio/blob/HEAD/gradio_app.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"849b90aa3c658242"}},{"code_sha256_prefix":"740af92e1f334629","entry":"predict_noise0_diffuser_multistep","repo":"yuanzhi-zhu/prolific_dreamer2d","repo_kind":"listed","path":"model_utils.py","file_url":"https://github.com/yuanzhi-zhu/prolific_dreamer2d/blob/HEAD/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"740af92e1f334629"}},{"code_sha256_prefix":"1b1adaf9115ca2a2","entry":"sds_vsd_grad_diffuser","repo":"yuanzhi-zhu/prolific_dreamer2d","repo_kind":"listed","path":"model_utils.py","file_url":"https://github.com/yuanzhi-zhu/prolific_dreamer2d/blob/HEAD/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1b1adaf9115ca2a2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}