{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generic-3d-diffusion-adapter-using-controlled","title":"Generic 3D Diffusion Adapter Using Controlled Multi-View Editing","arxiv_id":"2403.12032","date":"2024-03-18","proceeding":null,"authors":["Hansheng Chen","Ruoxi Shi","Yulin Liu","Bokui Shen","Jiayuan Gu","Gordon Wetzstein","Hao Su","Leonidas Guibas"],"abstract":"Open-domain 3D object synthesis has been lagging behind image synthesis due to limited data and higher computational complexity. To bridge this gap, recent works have investigated multi-view diffusion but often fall short in either 3D consistency, visual quality, or efficiency. This paper proposes MVEdit, which functions as a 3D counterpart of SDEdit, employing ancestral sampling to jointly denoise multi-view images and output high-quality textured meshes. Built on off-the-shelf 2D diffusion models, MVEdit achieves 3D consistency through a training-free 3D Adapter, which lifts the 2D views of the last timestep into a coherent 3D representation, then conditions the 2D views of the next timestep using rendered views, without uncompromising visual quality. With an inference time of only 2-5 minutes, this framework achieves better trade-off between quality and speed than score distillation. MVEdit is highly versatile and extendable, with a wide range of applications including text/image-to-3D generation, 3D-to-3D editing, and high-quality texture synthesis. In particular, evaluations demonstrate state-of-the-art performance in both image-to-3D and text-guided texture generation tasks. Additionally, we introduce a method for fine-tuning 2D latent diffusion models on small 3D datasets with limited resources, enabling fast low-resolution text-to-3D initialization.","url_abs":"https://arxiv.org/abs/2403.12032v2","url_pdf":"https://arxiv.org/pdf/2403.12032v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generic-3d-diffusion-adapter-using-controlled","repo_url":"https://github.com/Lakonik/MVEdit","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"3d-generation","task_name":"3D Generation"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"image-to-3d","task_name":"Image to 3D"},{"task_slug":"text-to-3d","task_name":"Text to 3D"},{"task_slug":"texture-synthesis","task_name":"Texture Synthesis"}],"methods":[{"method_slug":"adapter","method_name":"Adapter"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.12032","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}