{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/key-locked-rank-one-editing-for-text-to-image","title":"Key-Locked Rank One Editing for Text-to-Image Personalization","arxiv_id":"2305.01644","date":"2023-05-02","proceeding":null,"authors":["Yoad Tewel","Rinon Gal","Gal Chechik","Yuval Atzmon"],"abstract":"Text-to-image models (T2I) offer a new level of flexibility by allowing users to guide the creative process through natural language. However, personalizing these models to align with user-provided visual concepts remains a challenging problem. The task of T2I personalization poses multiple hard challenges, such as maintaining high visual fidelity while allowing creative control, combining multiple personalized concepts in a single image, and keeping a small model size. We present Perfusion, a T2I personalization method that addresses these challenges using dynamic rank-1 updates to the underlying T2I model. Perfusion avoids overfitting by introducing a new mechanism that \"locks\" new concepts' cross-attention Keys to their superordinate category. Additionally, we develop a gated rank-1 approach that enables us to control the influence of a learned concept during inference time and to combine multiple concepts. This allows runtime-efficient balancing of visual-fidelity and textual-alignment with a single 100KB trained model, which is five orders of magnitude smaller than the current state of the art. Moreover, it can span different operating points across the Pareto front without additional training. Finally, we show that Perfusion outperforms strong baselines in both qualitative and quantitative terms. Importantly, key-locking leads to novel results compared to traditional approaches, allowing to portray personalized object interactions in unprecedented ways, even in one-shot settings.","url_abs":"https://arxiv.org/abs/2305.01644v2","url_pdf":"https://arxiv.org/pdf/2305.01644v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"key-locked-rank-one-editing-for-text-to-image","repo_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2305.01644","atlas_url":"https://app.syntology.ai/?focus=2305.01644","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.01644"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":13,"ran_honours":1,"unverified":4},"by_repo_kind":{"listed":{"samples":18,"ran":14,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b04722a19845dc90","entry":"analytic_kernel","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"ldm/modules/image_degradation/bsrgan.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/ldm/modules/image_degradation/bsrgan.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b04722a19845dc90"}},{"code_sha256_prefix":"5a0ca1bba88bb4b0","entry":"anisotropic_Gaussian","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"ldm/modules/image_degradation/bsrgan.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/ldm/modules/image_degradation/bsrgan.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5a0ca1bba88bb4b0"}},{"code_sha256_prefix":"5b4a0a0fc3e2a388","entry":"clip_load_untrained","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"clipseg/models/clipseg.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/clipseg/models/clipseg.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5b4a0a0fc3e2a388"}},{"code_sha256_prefix":"057d183d358d1e9a","entry":"cosine_warmup_lr","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"clipseg/training.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/clipseg/training.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"057d183d358d1e9a"}},{"code_sha256_prefix":"17efe0b1bfd9f343","entry":"denorm","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"clipseg/evaluation_utils.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/clipseg/evaluation_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"17efe0b1bfd9f343"}},{"code_sha256_prefix":"e1d523a6f8c92026","entry":"fast_iou_curve","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"clipseg/evaluation_utils.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/clipseg/evaluation_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e1d523a6f8c92026"}},{"code_sha256_prefix":"febf3c4380082c74","entry":"filter_args","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"clipseg/general_utils.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/clipseg/general_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"febf3c4380082c74"}},{"code_sha256_prefix":"c35bef083a3a63d9","entry":"get_attribute","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"clipseg/general_utils.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/clipseg/general_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c35bef083a3a63d9"}},{"code_sha256_prefix":"f9d4c247328f3337","entry":"get_image_paths","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"ldm/modules/image_degradation/utils_image.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/ldm/modules/image_degradation/utils_image.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9d4c247328f3337"}},{"code_sha256_prefix":"f4c0e5f857a103d5","entry":"get_prompt_list","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"clipseg/models/clipseg.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/clipseg/models/clipseg.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f4c0e5f857a103d5"}},{"code_sha256_prefix":"226f6afcd57ff476","entry":"is_image_file","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"ldm/modules/image_degradation/utils_image.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/ldm/modules/image_degradation/utils_image.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"226f6afcd57ff476"}},{"code_sha256_prefix":"04df7487a791a8a1","entry":"modcrop_np","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"ldm/modules/image_degradation/bsrgan.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/ldm/modules/image_degradation/bsrgan.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"04df7487a791a8a1"}},{"code_sha256_prefix":"151e32d2a7b8134d","entry":"norm","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"clipseg/evaluation_utils.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/clipseg/evaluation_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"151e32d2a7b8134d"}},{"code_sha256_prefix":"9ef76f9c72933f80","entry":"process_prompts","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"clipseg/models/vitseg.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/clipseg/models/vitseg.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9ef76f9c72933f80"}},{"code_sha256_prefix":"9682702dd7af11cd","entry":"forward_multihead_attention","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"clipseg/models/clipseg.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/clipseg/models/clipseg.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9682702dd7af11cd"}},{"code_sha256_prefix":"c78a6fe00b0a118f","entry":"get_cached_pascal_pfe","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"clipseg/score.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/clipseg/score.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c78a6fe00b0a118f"}},{"code_sha256_prefix":"bca46b85e101b36d","entry":"load_model","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"clipseg/general_utils.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/clipseg/general_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bca46b85e101b36d"}},{"code_sha256_prefix":"3984b3ec31e90ef0","entry":"patches_from_image","repo":"ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization","repo_kind":"listed","path":"ldm/modules/image_degradation/utils_image.py","file_url":"https://github.com/ChenDarYen/Key-Locked-Rank-One-Editing-for-Text-to-Image-Personalization/blob/HEAD/ldm/modules/image_degradation/utils_image.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3984b3ec31e90ef0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}