{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/zero-shot-image-editing-with-reference","title":"Zero-shot Image Editing with Reference Imitation","arxiv_id":"2406.07547","date":"2024-06-11","proceeding":null,"authors":["Xi Chen","Yutong Feng","Mengting Chen","Yiyang Wang","Shilong Zhang","Yu Liu","Yujun Shen","Hengshuang Zhao"],"abstract":"Image editing serves as a practical yet challenging task considering the diverse demands from users, where one of the hardest parts is to precisely describe how the edited image should look like. In this work, we present a new form of editing, termed imitative editing, to help users exercise their creativity more conveniently. Concretely, to edit an image region of interest, users are free to directly draw inspiration from some in-the-wild references (e.g., some relative pictures come across online), without having to cope with the fit between the reference and the source. Such a design requires the system to automatically figure out what to expect from the reference to perform the editing. For this purpose, we propose a generative training framework, dubbed MimicBrush, which randomly selects two frames from a video clip, masks some regions of one frame, and learns to recover the masked regions using the information from the other frame. That way, our model, developed from a diffusion prior, is able to capture the semantic correspondence between separate images in a self-supervised manner. We experimentally show the effectiveness of our method under various test cases as well as its superiority over existing alternatives. We also construct a benchmark to facilitate further research.","url_abs":"https://arxiv.org/abs/2406.07547v1","url_pdf":"https://arxiv.org/pdf/2406.07547v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"zero-shot-image-editing-with-reference","repo_url":"https://github.com/ali-vilab/MimicBrush","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"semantic-correspondence","task_name":"Semantic correspondence"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.07547","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.07547"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ali-vilab/MimicBrush","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":1,"ran":1,"unverified":11},"by_repo_kind":{"listed":{"samples":13,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5105747c2711b1cb","entry":"FeedForward","repo":"ali-vilab/MimicBrush","repo_kind":"listed","path":"mimicbrush/resampler.py","file_url":"https://github.com/ali-vilab/MimicBrush/blob/HEAD/mimicbrush/resampler.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5105747c2711b1cb"}},{"code_sha256_prefix":"8199644cda7be3f8","entry":"calc_mean_std","repo":"ali-vilab/MimicBrush","repo_kind":"listed","path":"models/ReferenceNet_attention.py","file_url":"https://github.com/ali-vilab/MimicBrush/blob/HEAD/models/ReferenceNet_attention.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8199644cda7be3f8"}},{"code_sha256_prefix":"44791985965ff2de","entry":"collage_region","repo":"ali-vilab/MimicBrush","repo_kind":"listed","path":"run_gradio3_demo.py","file_url":"https://github.com/ali-vilab/MimicBrush/blob/HEAD/run_gradio3_demo.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"44791985965ff2de"}},{"code_sha256_prefix":"bcc8b9cd94a83d38","entry":"gaussian_blure","repo":"ali-vilab/MimicBrush","repo_kind":"listed","path":"dataset/data_utils.py","file_url":"https://github.com/ali-vilab/MimicBrush/blob/HEAD/dataset/data_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bcc8b9cd94a83d38"}},{"code_sha256_prefix":"6dddd743cbcceb4f","entry":"get_SIFT","repo":"ali-vilab/MimicBrush","repo_kind":"listed","path":"dataset/data_utils.py","file_url":"https://github.com/ali-vilab/MimicBrush/blob/HEAD/dataset/data_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6dddd743cbcceb4f"}},{"code_sha256_prefix":"230588e8f1237737","entry":"masked_mean","repo":"ali-vilab/MimicBrush","repo_kind":"listed","path":"mimicbrush/resampler.py","file_url":"https://github.com/ali-vilab/MimicBrush/blob/HEAD/mimicbrush/resampler.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"230588e8f1237737"}},{"code_sha256_prefix":"4a9a64c1228de9c4","entry":"pad_img_to_square","repo":"ali-vilab/MimicBrush","repo_kind":"listed","path":"run_gradio3_demo.py","file_url":"https://github.com/ali-vilab/MimicBrush/blob/HEAD/run_gradio3_demo.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4a9a64c1228de9c4"}},{"code_sha256_prefix":"a159146ab7ba81f1","entry":"prepare_mask_and_masked_image","repo":"ali-vilab/MimicBrush","repo_kind":"listed","path":"models/pipeline_mimicbrush.py","file_url":"https://github.com/ali-vilab/MimicBrush/blob/HEAD/models/pipeline_mimicbrush.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a159146ab7ba81f1"}},{"code_sha256_prefix":"ba8f22021dec0132","entry":"random_mask","repo":"ali-vilab/MimicBrush","repo_kind":"listed","path":"dataset/data_utils.py","file_url":"https://github.com/ali-vilab/MimicBrush/blob/HEAD/dataset/data_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ba8f22021dec0132"}},{"code_sha256_prefix":"4cb2e2a2ca0bec9f","entry":"reshape_tensor","repo":"ali-vilab/MimicBrush","repo_kind":"listed","path":"mimicbrush/resampler.py","file_url":"https://github.com/ali-vilab/MimicBrush/blob/HEAD/mimicbrush/resampler.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4cb2e2a2ca0bec9f"}},{"code_sha256_prefix":"0818a167c749d6f4","entry":"resize_image_keep_aspect_ratio","repo":"ali-vilab/MimicBrush","repo_kind":"listed","path":"run_gradio3_demo.py","file_url":"https://github.com/ali-vilab/MimicBrush/blob/HEAD/run_gradio3_demo.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0818a167c749d6f4"}},{"code_sha256_prefix":"b1c3f0a29f2e6e9d","entry":"retrieve_latents","repo":"ali-vilab/MimicBrush","repo_kind":"listed","path":"models/pipeline_mimicbrush.py","file_url":"https://github.com/ali-vilab/MimicBrush/blob/HEAD/models/pipeline_mimicbrush.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b1c3f0a29f2e6e9d"}},{"code_sha256_prefix":"e55f24470648724c","entry":"torch_dfs","repo":"ali-vilab/MimicBrush","repo_kind":"listed","path":"models/ReferenceNet_attention.py","file_url":"https://github.com/ali-vilab/MimicBrush/blob/HEAD/models/ReferenceNet_attention.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e55f24470648724c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}