{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ttida-controllable-generative-data","title":"TTIDA: Controllable Generative Data Augmentation via Text-to-Text and Text-to-Image Models","arxiv_id":"2304.08821","date":"2023-04-18","proceeding":null,"authors":["Yuwei Yin","Jean Kaddour","Xiang Zhang","Yixin Nie","Zhenguang Liu","Lingpeng Kong","Qi Liu"],"abstract":"Data augmentation has been established as an efficacious approach to supplement useful information for low-resource datasets. Traditional augmentation techniques such as noise injection and image transformations have been widely used. In addition, generative data augmentation (GDA) has been shown to produce more diverse and flexible data. While generative adversarial networks (GANs) have been frequently used for GDA, they lack diversity and controllability compared to text-to-image diffusion models. In this paper, we propose TTIDA (Text-to-Text-to-Image Data Augmentation) to leverage the capabilities of large-scale pre-trained Text-to-Text (T2T) and Text-to-Image (T2I) generative models for data augmentation. By conditioning the T2I model on detailed descriptions produced by T2T models, we are able to generate photo-realistic labeled images in a flexible and controllable manner. Experiments on in-domain classification, cross-domain classification, and image captioning tasks show consistent improvements over other data augmentation baselines. Analytical studies in varied settings, including few-shot, long-tail, and adversarial, further reinforce the effectiveness of TTIDA in enhancing performance and increasing robustness.","url_abs":"https://arxiv.org/abs/2304.08821v1","url_pdf":"https://arxiv.org/pdf/2304.08821v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ttida-controllable-generative-data","repo_url":"https://github.com/yuweiyin/ttida","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"image-captioning","task_name":"Image Captioning"},{"task_slug":"domain-classification","task_name":"domain classification"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2304.08821","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.08821"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yuweiyin/ttida","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"unverified":7},"by_repo_kind":{"official":{"samples":9,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e20dd5102da3b050","entry":"make_master_params","repo":"yuweiyin/ttida","repo_kind":"official","path":"glide_finetune/fp16_util.py","file_url":"https://github.com/yuweiyin/ttida/blob/HEAD/glide_finetune/fp16_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e20dd5102da3b050"}},{"code_sha256_prefix":"64fff1e30802b815","entry":"unflatten_master_params","repo":"yuweiyin/ttida","repo_kind":"official","path":"glide_finetune/fp16_util.py","file_url":"https://github.com/yuweiyin/ttida/blob/HEAD/glide_finetune/fp16_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"64fff1e30802b815"}},{"code_sha256_prefix":"0f1ef58c47413e31","entry":"get_image_files_dict","repo":"yuweiyin/ttida","repo_kind":"official","path":"glide_finetune/loader.py","file_url":"https://github.com/yuweiyin/ttida/blob/HEAD/glide_finetune/loader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f1ef58c47413e31"}},{"code_sha256_prefix":"4a01f5607a55ea2f","entry":"get_param_groups_and_shapes","repo":"yuweiyin/ttida","repo_kind":"official","path":"glide_finetune/fp16_util.py","file_url":"https://github.com/yuweiyin/ttida/blob/HEAD/glide_finetune/fp16_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a01f5607a55ea2f"}},{"code_sha256_prefix":"63b930347ee7b06d","entry":"get_text_files_dict","repo":"yuweiyin/ttida","repo_kind":"official","path":"glide_finetune/loader.py","file_url":"https://github.com/yuweiyin/ttida/blob/HEAD/glide_finetune/loader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"63b930347ee7b06d"}},{"code_sha256_prefix":"373efaac682e04f5","entry":"pil_image_to_norm_tensor","repo":"yuweiyin/ttida","repo_kind":"official","path":"glide_finetune/train_util.py","file_url":"https://github.com/yuweiyin/ttida/blob/HEAD/glide_finetune/train_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"373efaac682e04f5"}},{"code_sha256_prefix":"233d5c80d2043015","entry":"pred_to_pil","repo":"yuweiyin/ttida","repo_kind":"official","path":"glide_finetune/train_util.py","file_url":"https://github.com/yuweiyin/ttida/blob/HEAD/glide_finetune/train_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"233d5c80d2043015"}},{"code_sha256_prefix":"e9fdb363312ea83e","entry":"random_resized_crop","repo":"yuweiyin/ttida","repo_kind":"official","path":"glide_finetune/loader.py","file_url":"https://github.com/yuweiyin/ttida/blob/HEAD/glide_finetune/loader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e9fdb363312ea83e"}},{"code_sha256_prefix":"e8e4d867d3febb69","entry":"resize_for_upsample","repo":"yuweiyin/ttida","repo_kind":"official","path":"glide_finetune/train_util.py","file_url":"https://github.com/yuweiyin/ttida/blob/HEAD/glide_finetune/train_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e8e4d867d3febb69"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}