{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lora-composer-leveraging-low-rank-adaptation","title":"LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Models","arxiv_id":"2403.11627","date":"2024-03-18","proceeding":null,"authors":["Yang Yang","Wen Wang","Liang Peng","Chaotian Song","Yao Chen","Hengjia Li","Xiaolong Yang","Qinglin Lu","Deng Cai","Boxi Wu","Wei Liu"],"abstract":"Customization generation techniques have significantly advanced the synthesis of specific concepts across varied contexts. Multi-concept customization emerges as the challenging task within this domain. Existing approaches often rely on training a fusion matrix of multiple Low-Rank Adaptations (LoRAs) to merge various concepts into a single image. However, we identify this straightforward method faces two major challenges: 1) concept confusion, where the model struggles to preserve distinct individual characteristics, and 2) concept vanishing, where the model fails to generate the intended subjects. To address these issues, we introduce LoRA-Composer, a training-free framework designed for seamlessly integrating multiple LoRAs, thereby enhancing the harmony among different concepts within generated images. LoRA-Composer addresses concept vanishing through concept injection constraints, enhancing concept visibility via an expanded cross-attention mechanism. To combat concept confusion, concept isolation constraints are introduced, refining the self-attention computation. Furthermore, latent re-initialization is proposed to effectively stimulate concept-specific latent within designated regions. Our extensive testing showcases a notable enhancement in LoRA-Composer's performance compared to standard baselines, especially when eliminating the image-based conditions like canny edge or pose estimations. Code is released at \\url{https://github.com/Young98CN/LoRA_Composer}","url_abs":"https://arxiv.org/abs/2403.11627v2","url_pdf":"https://arxiv.org/pdf/2403.11627v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lora-composer-leveraging-low-rank-adaptation","repo_url":"https://github.com/young98cn/lora_composer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"lora-composer-leveraging-low-rank-adaptation","repo_url":"https://github.com/young98cn/lora_composer_page","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.11627","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.11627"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/young98cn/lora_composer","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/young98cn/lora_composer_page","reach":{"status":"ok"}}],"summary":{"ran":5,"unverified":3},"by_repo_kind":{"official":{"samples":8,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"49c29096f4f620e9","entry":"draw_rectangle","repo":"young98cn/lora_composer","repo_kind":"official","path":"region_lora/utils/attn_util.py","file_url":"https://github.com/young98cn/lora_composer/blob/HEAD/region_lora/utils/attn_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"49c29096f4f620e9"}},{"code_sha256_prefix":"056a6e797cbc0f04","entry":"generate_gaussian_matrix","repo":"young98cn/lora_composer","repo_kind":"official","path":"region_lora/multi_lora_sample.py","file_url":"https://github.com/young98cn/lora_composer/blob/HEAD/region_lora/multi_lora_sample.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"056a6e797cbc0f04"}},{"code_sha256_prefix":"58181e7380207f46","entry":"lora_moved","repo":"young98cn/lora_composer","repo_kind":"official","path":"mixofshow/archs/lora_override.py","file_url":"https://github.com/young98cn/lora_composer/blob/HEAD/mixofshow/archs/lora_override.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"58181e7380207f46"}},{"code_sha256_prefix":"3fd71341dfc51281","entry":"preprocess","repo":"young98cn/lora_composer","repo_kind":"official","path":"region_lora/multi_lora_sample.py","file_url":"https://github.com/young98cn/lora_composer/blob/HEAD/region_lora/multi_lora_sample.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3fd71341dfc51281"}},{"code_sha256_prefix":"cbb03c8ff754e2d0","entry":"text_under_image","repo":"young98cn/lora_composer","repo_kind":"official","path":"region_lora/utils/attn_util.py","file_url":"https://github.com/young98cn/lora_composer/blob/HEAD/region_lora/utils/attn_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"cbb03c8ff754e2d0"}},{"code_sha256_prefix":"4ea31d866b78ba77","entry":"aggregate_attention","repo":"young98cn/lora_composer","repo_kind":"official","path":"region_lora/utils/attn_util.py","file_url":"https://github.com/young98cn/lora_composer/blob/HEAD/region_lora/utils/attn_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4ea31d866b78ba77"}},{"code_sha256_prefix":"687032a155aea28d","entry":"merge_pplus2sd_","repo":"young98cn/lora_composer","repo_kind":"official","path":"inference/region_lora_sample.py","file_url":"https://github.com/young98cn/lora_composer/blob/HEAD/inference/region_lora_sample.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"687032a155aea28d"}},{"code_sha256_prefix":"32b15995404f4d2b","entry":"replace_latents","repo":"young98cn/lora_composer","repo_kind":"official","path":"region_lora/multi_lora_sample.py","file_url":"https://github.com/young98cn/lora_composer/blob/HEAD/region_lora/multi_lora_sample.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"32b15995404f4d2b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}