{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/visual-style-prompting-with-swapping-self","title":"Visual Style Prompting with Swapping Self-Attention","arxiv_id":"2402.12974","date":"2024-02-20","proceeding":null,"authors":["Jaeseok Jeong","Junho Kim","Yunjey Choi","Gayoung Lee","Youngjung Uh"],"abstract":"In the evolving domain of text-to-image generation, diffusion models have emerged as powerful tools in content creation. Despite their remarkable capability, existing models still face challenges in achieving controlled generation with a consistent style, requiring costly fine-tuning or often inadequately transferring the visual elements due to content leakage. To address these challenges, we propose a novel approach, \\ours, to produce a diverse range of images while maintaining specific style elements and nuances. During the denoising process, we keep the query from original features while swapping the key and value with those from reference features in the late self-attention layers. This approach allows for the visual style prompting without any fine-tuning, ensuring that generated images maintain a faithful style. Through extensive evaluation across various styles and text prompts, our method demonstrates superiority over existing approaches, best reflecting the style of the references and ensuring that resulting images match the text prompts most accurately. Our project page is available https://curryjung.github.io/VisualStylePrompt/.","url_abs":"https://arxiv.org/abs/2402.12974v2","url_pdf":"https://arxiv.org/pdf/2402.12974v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"visual-style-prompting-with-swapping-self","repo_url":"https://github.com/naver-ai/Visual-Style-Prompting","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"style-transfer","task_name":"Style Transfer"},{"task_slug":"text-to-image-generation-1","task_name":"Text to Image Generation"},{"task_slug":"text-to-image-generation","task_name":"Text-to-Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.12974","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.12974"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/naver-ai/Visual-Style-Prompting","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":14,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":16,"ran":15,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8d0ce4c22eacc88f","entry":"calc_mean_std","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"pipelines/inverted_ve_pipeline.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/pipelines/inverted_ve_pipeline.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8d0ce4c22eacc88f"}},{"code_sha256_prefix":"94f2cc13633a334f","entry":"concat_first","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"pipelines/inverted_ve_pipeline.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/pipelines/inverted_ve_pipeline.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"94f2cc13633a334f"}},{"code_sha256_prefix":"91fbd520765aec84","entry":"create_image","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"visualize_attention_src/utils.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/visualize_attention_src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"91fbd520765aec84"}},{"code_sha256_prefix":"8a18eb1d9dfeff92","entry":"create_image_grid","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"visualize_attention_src/save_attn_map_script.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/visualize_attention_src/save_attn_map_script.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8a18eb1d9dfeff92"}},{"code_sha256_prefix":"840989d505e53791","entry":"create_nested_list","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"vsp_real_script.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/vsp_real_script.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"840989d505e53791"}},{"code_sha256_prefix":"a9df142d09608ec2","entry":"create_number_list","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"vsp_real_script.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/vsp_real_script.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a9df142d09608ec2"}},{"code_sha256_prefix":"56be14bdb80a0850","entry":"create_prompt","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"vsp_real_script.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/vsp_real_script.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"56be14bdb80a0850"}},{"code_sha256_prefix":"197b9e6865c65770","entry":"expand_first","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"pipelines/inverted_ve_pipeline.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/pipelines/inverted_ve_pipeline.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"197b9e6865c65770"}},{"code_sha256_prefix":"93d7815126d56733","entry":"get_image","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"visualize_attention_src/utils.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/visualize_attention_src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"93d7815126d56733"}},{"code_sha256_prefix":"a9908edbd73a4ea0","entry":"get_image_v2","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"visualize_attention_src/utils.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/visualize_attention_src/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a9908edbd73a4ea0"}},{"code_sha256_prefix":"05ba22d160ba005c","entry":"init_latent","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"utils.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"05ba22d160ba005c"}},{"code_sha256_prefix":"8c171e27e9c1a1d0","entry":"load_config","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"utils.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8c171e27e9c1a1d0"}},{"code_sha256_prefix":"6b7da58f659741f1","entry":"parse_config","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"utils.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6b7da58f659741f1"}},{"code_sha256_prefix":"bea2d776a332f2b0","entry":"rescale_noise_cfg","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"visualize_attention_src/pipeline_stable_diffusion_xl_attn.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/visualize_attention_src/pipeline_stable_diffusion_xl_attn.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bea2d776a332f2b0"}},{"code_sha256_prefix":"9483beae09eebf51","entry":"transform_variable_name","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"visualize_attention_src/save_attn_map_script.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/visualize_attention_src/save_attn_map_script.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9483beae09eebf51"}},{"code_sha256_prefix":"da94debb8019ad46","entry":"zero_module","repo":"naver-ai/Visual-Style-Prompting","repo_kind":"official","path":"pipelines/controlnet.py","file_url":"https://github.com/naver-ai/Visual-Style-Prompting/blob/HEAD/pipelines/controlnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"da94debb8019ad46"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}