{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ms-diffusion-multi-subject-zero-shot-image","title":"MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidance","arxiv_id":"2406.07209","date":"2024-06-11","proceeding":null,"authors":["X. Wang","Siming Fu","Qihan Huang","Wanggui He","Hao Jiang"],"abstract":"Recent advancements in text-to-image generation models have dramatically enhanced the generation of photorealistic images from textual prompts, leading to an increased interest in personalized text-to-image applications, particularly in multi-subject scenarios. However, these advances are hindered by two main challenges: firstly, the need to accurately maintain the details of each referenced subject in accordance with the textual descriptions; and secondly, the difficulty in achieving a cohesive representation of multiple subjects in a single image without introducing inconsistencies. To address these concerns, our research introduces the MS-Diffusion framework for layout-guided zero-shot image personalization with multi-subjects. This innovative approach integrates grounding tokens with the feature resampler to maintain detail fidelity among subjects. With the layout guidance, MS-Diffusion further improves the cross-attention to adapt to the multi-subject inputs, ensuring that each subject condition acts on specific areas. The proposed multi-subject cross-attention orchestrates harmonious inter-subject compositions while preserving the control of texts. Comprehensive quantitative and qualitative experiments affirm that this method surpasses existing models in both image and text fidelity, promoting the development of personalized text-to-image generation.","url_abs":"https://arxiv.org/abs/2406.07209v1","url_pdf":"https://arxiv.org/pdf/2406.07209v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ms-diffusion-multi-subject-zero-shot-image","repo_url":"https://github.com/MS-Diffusion/MS-Diffusion","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"text-to-image-generation-1","task_name":"Text to Image Generation"},{"task_slug":"text-to-image-generation","task_name":"Text-to-Image Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.07209","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.07209"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MS-Diffusion/MS-Diffusion","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran":4,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5105747c2711b1cb","entry":"FeedForward","repo":"MS-Diffusion/MS-Diffusion","repo_kind":"official","path":"msdiffusion/models/projection.py","file_url":"https://github.com/MS-Diffusion/MS-Diffusion/blob/HEAD/msdiffusion/models/projection.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5105747c2711b1cb"}},{"code_sha256_prefix":"27c32a7bd4cb34f6","entry":"get_eot_idx","repo":"MS-Diffusion/MS-Diffusion","repo_kind":"official","path":"msdiffusion/utils.py","file_url":"https://github.com/MS-Diffusion/MS-Diffusion/blob/HEAD/msdiffusion/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"27c32a7bd4cb34f6"}},{"code_sha256_prefix":"4df68058ed6a2d10","entry":"get_phrase_idx","repo":"MS-Diffusion/MS-Diffusion","repo_kind":"official","path":"msdiffusion/utils.py","file_url":"https://github.com/MS-Diffusion/MS-Diffusion/blob/HEAD/msdiffusion/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4df68058ed6a2d10"}},{"code_sha256_prefix":"7b641c17c4887779","entry":"get_word_idx","repo":"MS-Diffusion/MS-Diffusion","repo_kind":"official","path":"msdiffusion/dataset/datagenerator.py","file_url":"https://github.com/MS-Diffusion/MS-Diffusion/blob/HEAD/msdiffusion/dataset/datagenerator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7b641c17c4887779"}},{"code_sha256_prefix":"423750878c152e2a","entry":"minmax_normalize","repo":"MS-Diffusion/MS-Diffusion","repo_kind":"official","path":"msdiffusion/models/attention_processor.py","file_url":"https://github.com/MS-Diffusion/MS-Diffusion/blob/HEAD/msdiffusion/models/attention_processor.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"423750878c152e2a"}},{"code_sha256_prefix":"230588e8f1237737","entry":"masked_mean","repo":"MS-Diffusion/MS-Diffusion","repo_kind":"official","path":"msdiffusion/models/projection.py","file_url":"https://github.com/MS-Diffusion/MS-Diffusion/blob/HEAD/msdiffusion/models/projection.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"230588e8f1237737"}},{"code_sha256_prefix":"4cb2e2a2ca0bec9f","entry":"reshape_tensor","repo":"MS-Diffusion/MS-Diffusion","repo_kind":"official","path":"msdiffusion/models/projection.py","file_url":"https://github.com/MS-Diffusion/MS-Diffusion/blob/HEAD/msdiffusion/models/projection.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4cb2e2a2ca0bec9f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}