{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/stylediffusion-prompt-embedding-inversion-for","title":"StyleDiffusion: Prompt-Embedding Inversion for Text-Based Editing","arxiv_id":"2303.15649","date":"2023-03-28","proceeding":null,"authors":["Senmao Li","Joost Van de Weijer","Taihang Hu","Fahad Shahbaz Khan","Qibin Hou","Yaxing Wang","Jian Yang","Ming-Ming Cheng"],"abstract":"A significant research effort is focused on exploiting the amazing capacities of pretrained diffusion models for the editing of images.They either finetune the model, or invert the image in the latent space of the pretrained model. However, they suffer from two problems: (1) Unsatisfying results for selected regions and unexpected changes in non-selected regions.(2) They require careful text prompt editing where the prompt should include all visual objects in the input image.To address this, we propose two improvements: (1) Only optimizing the input of the value linear network in the cross-attention layers is sufficiently powerful to reconstruct a real image. (2) We propose attention regularization to preserve the object-like attention maps after reconstruction and editing, enabling us to obtain accurate style editing without invoking significant structural changes. We further improve the editing technique that is used for the unconditional branch of classifier-free guidance as used by P2P. Extensive experimental prompt-editing results on a variety of images demonstrate qualitatively and quantitatively that our method has superior editing capabilities compared to existing and concurrent works. See our accompanying code in Stylediffusion: \\url{https://github.com/sen-mao/StyleDiffusion}.","url_abs":"https://arxiv.org/abs/2303.15649v3","url_pdf":"https://arxiv.org/pdf/2303.15649v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"stylediffusion-prompt-embedding-inversion-for","repo_url":"https://github.com/sen-mao/StyleDiffusion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"text-based-image-editing","task_name":"Text-based Image Editing"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/text-based-image-editing-on-pie-bench","task":"Text-based Image Editing","dataset":"PIE-Bench","model":"StyleDiffusion+Prompt-to-Prompt","rank_in_archive_order":9,"of":18,"metrics":{"Background LPIPS":"66.10","Background PSNR":"26.05","CLIPSIM":"24.78","Structure Distance":"11.65"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2303.15649","atlas_url":"https://app.syntology.ai/?focus=2303.15649","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.15649"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sen-mao/StyleDiffusion","reach":null}],"summary":{"ran_violates":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"b4991c55e7f549d1","entry":"str2bool","repo":"sen-mao/StyleDiffusion","repo_kind":"official","path":"Jittor/stylediffusion_csv.py","file_url":"https://github.com/sen-mao/StyleDiffusion/blob/HEAD/Jittor/stylediffusion_csv.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b4991c55e7f549d1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}