{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/viewcrafter-taming-video-diffusion-models-for","title":"ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis","arxiv_id":"2409.02048","date":"2024-09-03","proceeding":null,"authors":["Wangbo Yu","Jinbo Xing","Li Yuan","WenBo Hu","Xiaoyu Li","Zhipeng Huang","Xiangjun Gao","Tien-Tsin Wong","Ying Shan","Yonghong Tian"],"abstract":"Despite recent advancements in neural 3D reconstruction, the dependence on dense multi-view captures restricts their broader applicability. In this work, we propose \\textbf{ViewCrafter}, a novel method for synthesizing high-fidelity novel views of generic scenes from single or sparse images with the prior of video diffusion model. Our method takes advantage of the powerful generation capabilities of video diffusion model and the coarse 3D clues offered by point-based representation to generate high-quality video frames with precise camera pose control. To further enlarge the generation range of novel views, we tailored an iterative view synthesis strategy together with a camera trajectory planning algorithm to progressively extend the 3D clues and the areas covered by the novel views. With ViewCrafter, we can facilitate various applications, such as immersive experiences with real-time rendering by efficiently optimizing a 3D-GS representation using the reconstructed 3D points and the generated novel views, and scene-level text-to-3D generation for more imaginative content creation. Extensive experiments on diverse datasets demonstrate the strong generalization capability and superior performance of our method in synthesizing high-fidelity and consistent novel views.","url_abs":"https://arxiv.org/abs/2409.02048v1","url_pdf":"https://arxiv.org/pdf/2409.02048v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"viewcrafter-taming-video-diffusion-models-for","repo_url":"https://github.com/drexubery/viewcrafter","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"3d-generation","task_name":"3D Generation"},{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"},{"task_slug":"novel-view-synthesis","task_name":"Novel View Synthesis"},{"task_slug":"text-to-3d","task_name":"Text to 3D"},{"task_slug":"trajectory-planning","task_name":"Trajectory Planning"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2409.02048","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.02048"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/drexubery/viewcrafter","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_violates":1,"ran_draft_wrong":2,"ran_honours":1,"ran":1,"ran_fixture":1,"unverified":4},"by_repo_kind":{"listed":{"samples":10,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4cb732f513d69dfd","entry":"disabled_train","repo":"drexubery/viewcrafter","repo_kind":"listed","path":"lvdm/basics.py","file_url":"https://github.com/drexubery/viewcrafter/blob/HEAD/lvdm/basics.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4cb732f513d69dfd"}},{"code_sha256_prefix":"a15cfd7932844ef9","entry":"extract_into_tensor","repo":"drexubery/viewcrafter","repo_kind":"listed","path":"lvdm/common.py","file_url":"https://github.com/drexubery/viewcrafter/blob/HEAD/lvdm/common.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a15cfd7932844ef9"}},{"code_sha256_prefix":"c984209a2c66afbe","entry":"make_beta_schedule","repo":"drexubery/viewcrafter","repo_kind":"listed","path":"lvdm/models/utils_diffusion.py","file_url":"https://github.com/drexubery/viewcrafter/blob/HEAD/lvdm/models/utils_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c984209a2c66afbe"}},{"code_sha256_prefix":"17eb0a31c91b671c","entry":"normal_kl","repo":"drexubery/viewcrafter","repo_kind":"listed","path":"lvdm/distributions.py","file_url":"https://github.com/drexubery/viewcrafter/blob/HEAD/lvdm/distributions.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"17eb0a31c91b671c"}},{"code_sha256_prefix":"7e83c65f054fb972","entry":"timestep_embedding","repo":"drexubery/viewcrafter","repo_kind":"listed","path":"lvdm/models/utils_diffusion.py","file_url":"https://github.com/drexubery/viewcrafter/blob/HEAD/lvdm/models/utils_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7e83c65f054fb972"}},{"code_sha256_prefix":"129b804760b3115f","entry":"zero_module","repo":"drexubery/viewcrafter","repo_kind":"listed","path":"lvdm/basics.py","file_url":"https://github.com/drexubery/viewcrafter/blob/HEAD/lvdm/basics.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"129b804760b3115f"}},{"code_sha256_prefix":"32af1c12fd826ef2","entry":"autocast","repo":"drexubery/viewcrafter","repo_kind":"listed","path":"lvdm/common.py","file_url":"https://github.com/drexubery/viewcrafter/blob/HEAD/lvdm/common.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"32af1c12fd826ef2"}},{"code_sha256_prefix":"b72d8eb89fd75383","entry":"gather_data","repo":"drexubery/viewcrafter","repo_kind":"listed","path":"lvdm/common.py","file_url":"https://github.com/drexubery/viewcrafter/blob/HEAD/lvdm/common.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b72d8eb89fd75383"}},{"code_sha256_prefix":"662fc3ac21ada8a7","entry":"make_ddim_timesteps","repo":"drexubery/viewcrafter","repo_kind":"listed","path":"lvdm/models/utils_diffusion.py","file_url":"https://github.com/drexubery/viewcrafter/blob/HEAD/lvdm/models/utils_diffusion.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"662fc3ac21ada8a7"}},{"code_sha256_prefix":"9136b5f9b735334b","entry":"scale_module","repo":"drexubery/viewcrafter","repo_kind":"listed","path":"lvdm/basics.py","file_url":"https://github.com/drexubery/viewcrafter/blob/HEAD/lvdm/basics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9136b5f9b735334b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}