{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/omg-occlusion-friendly-personalized-multi","title":"OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models","arxiv_id":"2403.10983","date":"2024-03-16","proceeding":null,"authors":["Zhe Kong","Yong Zhang","Tianyu Yang","Tao Wang","Kaihao Zhang","Bizhu Wu","GuanYing Chen","Wei Liu","Wenhan Luo"],"abstract":"Personalization is an important topic in text-to-image generation, especially the challenging multi-concept personalization. Current multi-concept methods are struggling with identity preservation, occlusion, and the harmony between foreground and background. In this work, we propose OMG, an occlusion-friendly personalized generation framework designed to seamlessly integrate multiple concepts within a single image. We propose a novel two-stage sampling solution. The first stage takes charge of layout generation and visual comprehension information collection for handling occlusions. The second one utilizes the acquired visual comprehension information and the designed noise blending to integrate multiple concepts while considering occlusions. We also observe that the initiation denoising timestep for noise blending is the key to identity preservation and layout. Moreover, our method can be combined with various single-concept models, such as LoRA and InstantID without additional tuning. Especially, LoRA models on civitai.com can be exploited directly. Extensive experiments demonstrate that OMG exhibits superior performance in multi-concept personalization.","url_abs":"https://arxiv.org/abs/2403.10983v2","url_pdf":"https://arxiv.org/pdf/2403.10983v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"omg-occlusion-friendly-personalized-multi","repo_url":"https://github.com/kongzhecn/omg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"layout-generation","task_name":"Layout Generation"},{"task_slug":"text-to-image-generation-1","task_name":"Text to Image Generation"},{"task_slug":"text-to-image-generation","task_name":"Text-to-Image Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.10983","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.10983"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kongzhecn/omg","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"ran":4,"unverified":8},"by_repo_kind":{"official":{"samples":13,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":13,"samples":[{"code_sha256_prefix":"5105747c2711b1cb","entry":"FeedForward","repo":"kongzhecn/omg","repo_kind":"official","path":"src/ip_adapter/resampler.py","file_url":"https://github.com/kongzhecn/omg/blob/HEAD/src/ip_adapter/resampler.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5105747c2711b1cb"}},{"code_sha256_prefix":"153b62620c0ab2df","entry":"draw_kps","repo":"kongzhecn/omg","repo_kind":"official","path":"src/pipelines/instantid_single_pieline.py","file_url":"https://github.com/kongzhecn/omg/blob/HEAD/src/pipelines/instantid_single_pieline.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"153b62620c0ab2df"}},{"code_sha256_prefix":"19b2819e7636c67b","entry":"load_image_yoloworld","repo":"kongzhecn/omg","repo_kind":"official","path":"inference_lora.py","file_url":"https://github.com/kongzhecn/omg/blob/HEAD/inference_lora.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19b2819e7636c67b"}},{"code_sha256_prefix":"c388d6c8526b0dda","entry":"load_image_yoloworld","repo":"kongzhecn/omg","repo_kind":"official","path":"gradio_demo/app_generateOne.py","file_url":"https://github.com/kongzhecn/omg/blob/HEAD/gradio_demo/app_generateOne.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c388d6c8526b0dda"}},{"code_sha256_prefix":"a14ba604db525034","entry":"sample_image","repo":"kongzhecn/omg","repo_kind":"official","path":"inference_lora.py","file_url":"https://github.com/kongzhecn/omg/blob/HEAD/inference_lora.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a14ba604db525034"}},{"code_sha256_prefix":"a120d749646dc503","entry":"build_dino_segment_model","repo":"kongzhecn/omg","repo_kind":"official","path":"inference_instantid.py","file_url":"https://github.com/kongzhecn/omg/blob/HEAD/inference_instantid.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a120d749646dc503"}},{"code_sha256_prefix":"fe655f9336d78bf3","entry":"build_yolo_segment_model","repo":"kongzhecn/omg","repo_kind":"official","path":"inference_instantid.py","file_url":"https://github.com/kongzhecn/omg/blob/HEAD/inference_instantid.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fe655f9336d78bf3"}},{"code_sha256_prefix":"ee958d65bb51ac3d","entry":"get_face_embedding","repo":"kongzhecn/omg","repo_kind":"official","path":"src/pipelines/instantid_pipeline.py","file_url":"https://github.com/kongzhecn/omg/blob/HEAD/src/pipelines/instantid_pipeline.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ee958d65bb51ac3d"}},{"code_sha256_prefix":"aac6c0d2c1aa8b34","entry":"load_image_dino","repo":"kongzhecn/omg","repo_kind":"official","path":"inference_lora.py","file_url":"https://github.com/kongzhecn/omg/blob/HEAD/inference_lora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aac6c0d2c1aa8b34"}},{"code_sha256_prefix":"0894401e52396ab0","entry":"load_image_dino","repo":"kongzhecn/omg","repo_kind":"official","path":"gradio_demo/app_generateOne.py","file_url":"https://github.com/kongzhecn/omg/blob/HEAD/gradio_demo/app_generateOne.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0894401e52396ab0"}},{"code_sha256_prefix":"cc4a3e23216117ff","entry":"load_model_hf","repo":"kongzhecn/omg","repo_kind":"official","path":"inference_instantid.py","file_url":"https://github.com/kongzhecn/omg/blob/HEAD/inference_instantid.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cc4a3e23216117ff"}},{"code_sha256_prefix":"4cb2e2a2ca0bec9f","entry":"reshape_tensor","repo":"kongzhecn/omg","repo_kind":"official","path":"src/ip_adapter/resampler.py","file_url":"https://github.com/kongzhecn/omg/blob/HEAD/src/ip_adapter/resampler.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4cb2e2a2ca0bec9f"}},{"code_sha256_prefix":"360b96a50c9eec81","entry":"revise_regionally_controlnet_forward","repo":"kongzhecn/omg","repo_kind":"official","path":"src/pipelines/instantid_pipeline.py","file_url":"https://github.com/kongzhecn/omg/blob/HEAD/src/pipelines/instantid_pipeline.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"360b96a50c9eec81"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}