{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sine-single-image-editing-with-text-to-image","title":"SINE: SINgle Image Editing with Text-to-Image Diffusion Models","arxiv_id":"2212.04489","date":"2022-12-08","proceeding":"CVPR 2023 1","authors":["Zhixing Zhang","Ligong Han","Arnab Ghosh","Dimitris Metaxas","Jian Ren"],"abstract":"Recent works on diffusion models have demonstrated a strong capability for conditioning image generation, e.g., text-guided image synthesis. Such success inspires many efforts trying to use large-scale pre-trained diffusion models for tackling a challenging problem--real image editing. Works conducted in this area learn a unique textual token corresponding to several images containing the same object. However, under many circumstances, only one image is available, such as the painting of the Girl with a Pearl Earring. Using existing works on fine-tuning the pre-trained diffusion models with a single image causes severe overfitting issues. The information leakage from the pre-trained diffusion models makes editing can not keep the same content as the given image while creating new features depicted by the language guidance. This work aims to address the problem of single-image editing. We propose a novel model-based guidance built upon the classifier-free guidance so that the knowledge from the model trained on a single image can be distilled into the pre-trained diffusion model, enabling content creation even with one given image. Additionally, we propose a patch-based fine-tuning that can effectively help the model generate images of arbitrary resolution. We provide extensive experiments to validate the design choices of our approach and show promising editing capabilities, including changing style, content addition, and object manipulation. The code is available for research purposes at https://github.com/zhang-zx/SINE.git .","url_abs":"https://arxiv.org/abs/2212.04489v2","url_pdf":"https://arxiv.org/pdf/2212.04489v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sine-single-image-editing-with-text-to-image","repo_url":"https://github.com/zhang-zx/sine","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2212.04489","atlas_url":"https://app.syntology.ai/?focus=2212.04489","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.04489"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhang-zx/sine","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":3,"ran":4,"ran_violates":3,"ran_honours":2,"unverified":3},"by_repo_kind":{"official":{"samples":15,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fe5dd5258046898c","entry":"always","repo":"zhang-zx/sine","repo_kind":"official","path":"ldm/modules/x_transformer.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/ldm/modules/x_transformer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe5dd5258046898c"}},{"code_sha256_prefix":"60c484c8af7847b0","entry":"collate_fn","repo":"zhang-zx/sine","repo_kind":"official","path":"diffusers_train.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/diffusers_train.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"60c484c8af7847b0"}},{"code_sha256_prefix":"424012cb37b31172","entry":"default","repo":"zhang-zx/sine","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"424012cb37b31172"}},{"code_sha256_prefix":"4cb732f513d69dfd","entry":"disabled_train","repo":"zhang-zx/sine","repo_kind":"official","path":"ldm/models/diffusion/classifier.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/ldm/models/diffusion/classifier.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4cb732f513d69dfd"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"zhang-zx/sine","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"d5726f9e3a5684c3","entry":"get_bert_token_for_string","repo":"zhang-zx/sine","repo_kind":"official","path":"ldm/modules/embedding_manager.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/ldm/modules/embedding_manager.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d5726f9e3a5684c3"}},{"code_sha256_prefix":"41c6eb3a0aa3eb46","entry":"get_clip_token_for_string","repo":"zhang-zx/sine","repo_kind":"official","path":"ldm/modules/embedding_manager.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/ldm/modules/embedding_manager.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"41c6eb3a0aa3eb46"}},{"code_sha256_prefix":"71c2df28a1008d03","entry":"get_embedding_for_clip_token","repo":"zhang-zx/sine","repo_kind":"official","path":"ldm/modules/embedding_manager.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/ldm/modules/embedding_manager.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"71c2df28a1008d03"}},{"code_sha256_prefix":"b1368330cf0f5642","entry":"isimage","repo":"zhang-zx/sine","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1368330cf0f5642"}},{"code_sha256_prefix":"d72762b700feee6f","entry":"ismap","repo":"zhang-zx/sine","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d72762b700feee6f"}},{"code_sha256_prefix":"79e68e0b49a43379","entry":"parse_args","repo":"zhang-zx/sine","repo_kind":"official","path":"diffusers_sample.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/diffusers_sample.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"79e68e0b49a43379"}},{"code_sha256_prefix":"9a299fe5ae09e407","entry":"uniq","repo":"zhang-zx/sine","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a299fe5ae09e407"}},{"code_sha256_prefix":"2f00b39db5a81466","entry":"import_model_class_from_model_name_or_path","repo":"zhang-zx/sine","repo_kind":"official","path":"diffusers_sample.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/diffusers_sample.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2f00b39db5a81466"}},{"code_sha256_prefix":"f9bd2e83191afad1","entry":"log_txt_as_img","repo":"zhang-zx/sine","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9bd2e83191afad1"}},{"code_sha256_prefix":"977f7fb7829cd878","entry":"parse_args","repo":"zhang-zx/sine","repo_kind":"official","path":"diffusers_train.py","file_url":"https://github.com/zhang-zx/sine/blob/HEAD/diffusers_train.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"977f7fb7829cd878"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}