{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dragdiffusion-harnessing-diffusion-models-for","title":"DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing","arxiv_id":"2306.14435","date":"2023-06-26","proceeding":"CVPR 2024 1","authors":["Yujun Shi","Chuhui Xue","Jun Hao Liew","Jiachun Pan","Hanshu Yan","Wenqing Zhang","Vincent Y. F. Tan","Song Bai"],"abstract":"Accurate and controllable image editing is a challenging task that has attracted significant attention recently. Notably, DragGAN is an interactive point-based image editing framework that achieves impressive editing results with pixel-level precision. However, due to its reliance on generative adversarial networks (GANs), its generality is limited by the capacity of pretrained GAN models. In this work, we extend this editing framework to diffusion models and propose a novel approach DragDiffusion. By harnessing large-scale pretrained diffusion models, we greatly enhance the applicability of interactive point-based editing on both real and diffusion-generated images. Our approach involves optimizing the diffusion latents to achieve precise spatial control. The supervision signal of this optimization process is from the diffusion model's UNet features, which are known to contain rich semantic and geometric information. Moreover, we introduce two additional techniques, namely LoRA fine-tuning and latent-MasaCtrl, to further preserve the identity of the original image. Lastly, we present a challenging benchmark dataset called DragBench -- the first benchmark to evaluate the performance of interactive point-based image editing methods. Experiments across a wide range of challenging cases (e.g., images with multiple objects, diverse object categories, various styles, etc.) demonstrate the versatility and generality of DragDiffusion. Code: https://github.com/Yujun-Shi/DragDiffusion.","url_abs":"https://arxiv.org/abs/2306.14435v6","url_pdf":"https://arxiv.org/pdf/2306.14435v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dragdiffusion-harnessing-diffusion-models-for","repo_url":"https://github.com/Yujun-Shi/DragDiffusion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"dragdiffusion-harnessing-diffusion-models-for","repo_url":"https://github.com/HimariO/DragDiffusion-diffusers","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"dragdiffusion-harnessing-diffusion-models-for","repo_url":"https://github.com/MICV-yonsei/DragText","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"dragdiffusion-harnessing-diffusion-models-for","repo_url":"https://github.com/tony-lowe/rotationdrag","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.14435","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.14435"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tony-lowe/rotationdrag","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Yujun-Shi/DragDiffusion","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/HimariO/DragDiffusion-diffusers","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MICV-yonsei/DragText","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":2,"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1},"listed":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a3379285d38d5ba2","entry":"backward_ddim","repo":"HimariO/DragDiffusion-diffusers","repo_kind":"listed","path":"drag_pipeline.py","file_url":"https://github.com/HimariO/DragDiffusion-diffusers/blob/HEAD/drag_pipeline.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a3379285d38d5ba2"}},{"code_sha256_prefix":"256040ccde7473f2","entry":"collate_fn","repo":"Yujun-Shi/DragDiffusion","repo_kind":"official","path":"lora/train_dreambooth_lora.py","file_url":"https://github.com/Yujun-Shi/DragDiffusion/blob/HEAD/lora/train_dreambooth_lora.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"256040ccde7473f2"}},{"code_sha256_prefix":"85fd2e71002a15f4","entry":"forward_ddim","repo":"HimariO/DragDiffusion-diffusers","repo_kind":"listed","path":"drag_pipeline.py","file_url":"https://github.com/HimariO/DragDiffusion-diffusers/blob/HEAD/drag_pipeline.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"85fd2e71002a15f4"}},{"code_sha256_prefix":"acae03ffff4ba768","entry":"load_img","repo":"HimariO/DragDiffusion-diffusers","repo_kind":"listed","path":"drag_pipeline.py","file_url":"https://github.com/HimariO/DragDiffusion-diffusers/blob/HEAD/drag_pipeline.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"acae03ffff4ba768"}},{"code_sha256_prefix":"ce6abf8518e8fd67","entry":"parse_args","repo":"Yujun-Shi/DragDiffusion","repo_kind":"official","path":"lora/train_dreambooth_lora.py","file_url":"https://github.com/Yujun-Shi/DragDiffusion/blob/HEAD/lora/train_dreambooth_lora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ce6abf8518e8fd67"}},{"code_sha256_prefix":"564d1e2f74eb0245","entry":"prepare_mask_and_masked_image","repo":"HimariO/DragDiffusion-diffusers","repo_kind":"listed","path":"drag_inpaint_pipeline.py","file_url":"https://github.com/HimariO/DragDiffusion-diffusers/blob/HEAD/drag_inpaint_pipeline.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"564d1e2f74eb0245"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}