{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/controlcom-controllable-image-composition","title":"ControlCom: Controllable Image Composition using Diffusion Model","arxiv_id":"2308.10040","date":"2023-08-19","proceeding":null,"authors":["Bo Zhang","Yuxuan Duan","Jun Lan","Yan Hong","Huijia Zhu","Weiqiang Wang","Li Niu"],"abstract":"Image composition targets at synthesizing a realistic composite image from a pair of foreground and background images. Recently, generative composition methods are built on large pretrained diffusion models to generate composite images, considering their great potential in image generation. However, they suffer from lack of controllability on foreground attributes and poor preservation of foreground identity. To address these challenges, we propose a controllable image composition method that unifies four tasks in one diffusion model: image blending, image harmonization, view synthesis, and generative composition. Meanwhile, we design a self-supervised training framework coupled with a tailored pipeline of training data preparation. Moreover, we propose a local enhancement module to enhance the foreground details in the diffusion model, improving the foreground fidelity of composite images. The proposed method is evaluated on both public benchmark and real-world data, which demonstrates that our method can generate more faithful and controllable composite images than existing approaches. The code and model will be available at https://github.com/bcmi/ControlCom-Image-Composition.","url_abs":"https://arxiv.org/abs/2308.10040v1","url_pdf":"https://arxiv.org/pdf/2308.10040v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"controlcom-controllable-image-composition","repo_url":"https://github.com/bcmi/controlcom-image-composition","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"image-harmonization","task_name":"Image Harmonization"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2308.10040","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.10040"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bcmi/controlcom-image-composition","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":6,"ran_honours":1,"ran_violates":4,"ran_fixture":1,"ran":2,"unverified":1},"by_repo_kind":{"official":{"samples":13,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"9fcdaa6e423e8aa7","entry":"Normalize","repo":"bcmi/controlcom-image-composition","repo_kind":"official","path":"ldm/modules/diffusionmodules/model.py","file_url":"https://github.com/bcmi/controlcom-image-composition/blob/HEAD/ldm/modules/diffusionmodules/model.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9fcdaa6e423e8aa7"}},{"code_sha256_prefix":"fe5dd5258046898c","entry":"always","repo":"bcmi/controlcom-image-composition","repo_kind":"official","path":"ldm/modules/x_transformer.py","file_url":"https://github.com/bcmi/controlcom-image-composition/blob/HEAD/ldm/modules/x_transformer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe5dd5258046898c"}},{"code_sha256_prefix":"8241c0562bc710fd","entry":"chunk","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"8241c0562bc710fd"}},{"code_sha256_prefix":"3e5f67215d50b497","entry":"clip2sd","repo":"bcmi/controlcom-image-composition","repo_kind":"official","path":"ldm/models/diffusion/ddpm.py","file_url":"https://github.com/bcmi/controlcom-image-composition/blob/HEAD/ldm/models/diffusion/ddpm.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e5f67215d50b497"}},{"code_sha256_prefix":"424012cb37b31172","entry":"default","repo":"bcmi/controlcom-image-composition","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/bcmi/controlcom-image-composition/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"424012cb37b31172"}},{"code_sha256_prefix":"4cb732f513d69dfd","entry":"disabled_train","repo":"bcmi/controlcom-image-composition","repo_kind":"official","path":"ldm/models/diffusion/classifier.py","file_url":"https://github.com/bcmi/controlcom-image-composition/blob/HEAD/ldm/models/diffusion/classifier.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4cb732f513d69dfd"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"bcmi/controlcom-image-composition","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/bcmi/controlcom-image-composition/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"cb49209c125de1b4","entry":"get_timestep_embedding","repo":"bcmi/controlcom-image-composition","repo_kind":"official","path":"ldm/modules/diffusionmodules/model.py","file_url":"https://github.com/bcmi/controlcom-image-composition/blob/HEAD/ldm/modules/diffusionmodules/model.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb49209c125de1b4"}},{"code_sha256_prefix":"b1368330cf0f5642","entry":"isimage","repo":"bcmi/controlcom-image-composition","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/bcmi/controlcom-image-composition/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1368330cf0f5642"}},{"code_sha256_prefix":"d72762b700feee6f","entry":"ismap","repo":"bcmi/controlcom-image-composition","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/bcmi/controlcom-image-composition/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d72762b700feee6f"}},{"code_sha256_prefix":"3137073275f8c21a","entry":"nonlinearity","repo":"bcmi/controlcom-image-composition","repo_kind":"official","path":"ldm/modules/diffusionmodules/model.py","file_url":"https://github.com/bcmi/controlcom-image-composition/blob/HEAD/ldm/modules/diffusionmodules/model.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3137073275f8c21a"}},{"code_sha256_prefix":"1e63d588563eb90a","entry":"numpy_to_pil","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"1e63d588563eb90a"}},{"code_sha256_prefix":"d48d8354986e3b0e","entry":"uniform_on_device","repo":"bcmi/controlcom-image-composition","repo_kind":"official","path":"ldm/models/diffusion/ddpm.py","file_url":"https://github.com/bcmi/controlcom-image-composition/blob/HEAD/ldm/models/diffusion/ddpm.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d48d8354986e3b0e"}},{"code_sha256_prefix":"9a299fe5ae09e407","entry":"uniq","repo":"bcmi/controlcom-image-composition","repo_kind":"official","path":"ldm/modules/attention.py","file_url":"https://github.com/bcmi/controlcom-image-composition/blob/HEAD/ldm/modules/attention.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a299fe5ae09e407"}},{"code_sha256_prefix":"f9bd2e83191afad1","entry":"log_txt_as_img","repo":"bcmi/controlcom-image-composition","repo_kind":"official","path":"ldm/util.py","file_url":"https://github.com/bcmi/controlcom-image-composition/blob/HEAD/ldm/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f9bd2e83191afad1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}