{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gen4gen-generative-data-pipeline-for","title":"Gen4Gen: Generative Data Pipeline for Generative Multi-Concept Composition","arxiv_id":"2402.15504","date":"2024-02-23","proceeding":null,"authors":["Chun-Hsiao Yeh","Ta-Ying Cheng","He-Yen Hsieh","Chuan-En Lin","Yi Ma","Andrew Markham","Niki Trigoni","H. T. Kung","Yubei Chen"],"abstract":"Recent text-to-image diffusion models are able to learn and synthesize images containing novel, personalized concepts (e.g., their own pets or specific items) with just a few examples for training. This paper tackles two interconnected issues within this realm of personalizing text-to-image diffusion models. First, current personalization techniques fail to reliably extend to multiple concepts -- we hypothesize this to be due to the mismatch between complex scenes and simple text descriptions in the pre-training dataset (e.g., LAION). Second, given an image containing multiple personalized concepts, there lacks a holistic metric that evaluates performance on not just the degree of resemblance of personalized concepts, but also whether all concepts are present in the image and whether the image accurately reflects the overall text description. To address these issues, we introduce Gen4Gen, a semi-automated dataset creation pipeline utilizing generative models to combine personalized concepts into complex compositions along with text-descriptions. Using this, we create a dataset called MyCanvas, that can be used to benchmark the task of multi-concept personalization. In addition, we design a comprehensive metric comprising two scores (CP-CLIP and TI-CLIP) for better quantifying the performance of multi-concept, personalized text-to-image diffusion methods. We provide a simple baseline built on top of Custom Diffusion with empirical prompting strategies for future researchers to evaluate on MyCanvas. We show that by improving data quality and prompting strategies, we can significantly increase multi-concept personalized image generation quality, without requiring any modifications to model architecture or training algorithms.","url_abs":"https://arxiv.org/abs/2402.15504v1","url_pdf":"https://arxiv.org/pdf/2402.15504v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gen4gen-generative-data-pipeline-for","repo_url":"https://github.com/louisYen/Gen4Gen","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"personalized-image-generation","task_name":"Personalized Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.15504","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.15504"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/louisYen/Gen4Gen","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":8,"unverified":2},"by_repo_kind":{"official":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"729599a34939c1bf","entry":"approximate_RDP","repo":"louisYen/Gen4Gen","repo_kind":"official","path":"gen4gen/saliency_models/DIS/hce_metric_main.py","file_url":"https://github.com/louisYen/Gen4Gen/blob/HEAD/gen4gen/saliency_models/DIS/hce_metric_main.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"729599a34939c1bf"}},{"code_sha256_prefix":"ccf648472bf97751","entry":"f1score_torch","repo":"louisYen/Gen4Gen","repo_kind":"official","path":"gen4gen/saliency_models/DIS/basics.py","file_url":"https://github.com/louisYen/Gen4Gen/blob/HEAD/gen4gen/saliency_models/DIS/basics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ccf648472bf97751"}},{"code_sha256_prefix":"b1f805b5d053ac0d","entry":"filter_bdy_cond","repo":"louisYen/Gen4Gen","repo_kind":"official","path":"gen4gen/saliency_models/DIS/hce_metric_main.py","file_url":"https://github.com/louisYen/Gen4Gen/blob/HEAD/gen4gen/saliency_models/DIS/hce_metric_main.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1f805b5d053ac0d"}},{"code_sha256_prefix":"c3c043344336bb0b","entry":"get_im_gt_name_dict","repo":"louisYen/Gen4Gen","repo_kind":"official","path":"gen4gen/saliency_models/DIS/data_loader_cache.py","file_url":"https://github.com/louisYen/Gen4Gen/blob/HEAD/gen4gen/saliency_models/DIS/data_loader_cache.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c3c043344336bb0b"}},{"code_sha256_prefix":"2ebc4e2ea773d416","entry":"get_objects","repo":"louisYen/Gen4Gen","repo_kind":"official","path":"gen4gen/s2_llm_guided_object_composition.py","file_url":"https://github.com/louisYen/Gen4Gen/blob/HEAD/gen4gen/s2_llm_guided_object_composition.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2ebc4e2ea773d416"}},{"code_sha256_prefix":"a761df1609ffb30b","entry":"im_reader","repo":"louisYen/Gen4Gen","repo_kind":"official","path":"gen4gen/saliency_models/DIS/data_loader_cache.py","file_url":"https://github.com/louisYen/Gen4Gen/blob/HEAD/gen4gen/saliency_models/DIS/data_loader_cache.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a761df1609ffb30b"}},{"code_sha256_prefix":"1ad7043f7cda3831","entry":"mae_torch","repo":"louisYen/Gen4Gen","repo_kind":"official","path":"gen4gen/saliency_models/DIS/basics.py","file_url":"https://github.com/louisYen/Gen4Gen/blob/HEAD/gen4gen/saliency_models/DIS/basics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ad7043f7cda3831"}},{"code_sha256_prefix":"de8843c32f8cfda5","entry":"relax_HCE","repo":"louisYen/Gen4Gen","repo_kind":"official","path":"gen4gen/saliency_models/DIS/hce_metric_main.py","file_url":"https://github.com/louisYen/Gen4Gen/blob/HEAD/gen4gen/saliency_models/DIS/hce_metric_main.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de8843c32f8cfda5"}},{"code_sha256_prefix":"5cd2aa75a7b0c6ec","entry":"create_dataloaders","repo":"louisYen/Gen4Gen","repo_kind":"official","path":"gen4gen/saliency_models/DIS/data_loader_cache.py","file_url":"https://github.com/louisYen/Gen4Gen/blob/HEAD/gen4gen/saliency_models/DIS/data_loader_cache.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5cd2aa75a7b0c6ec"}},{"code_sha256_prefix":"7d13248918a66377","entry":"f1_mae_torch","repo":"louisYen/Gen4Gen","repo_kind":"official","path":"gen4gen/saliency_models/DIS/basics.py","file_url":"https://github.com/louisYen/Gen4Gen/blob/HEAD/gen4gen/saliency_models/DIS/basics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7d13248918a66377"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}