{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/data-extrapolation-for-text-to-image","title":"Data Extrapolation for Text-to-image Generation on Small Datasets","arxiv_id":"2410.01638","date":"2024-10-02","proceeding":null,"authors":["Senmao Ye","Fei Liu"],"abstract":"Text-to-image generation requires large amount of training data to synthesizing high-quality images. For augmenting training data, previous methods rely on data interpolations like cropping, flipping, and mixing up, which fail to introduce new information and yield only marginal improvements. In this paper, we propose a new data augmentation method for text-to-image generation using linear extrapolation. Specifically, we apply linear extrapolation only on text feature, and new image data are retrieved from the internet by search engines. For the reliability of new text-image pairs, we design two outlier detectors to purify retrieved images. Based on extrapolation, we construct training samples dozens of times larger than the original dataset, resulting in a significant improvement in text-to-image performance. Moreover, we propose a NULL-guidance to refine score estimation, and apply recurrent affine transformation to fuse text information. Our model achieves FID scores of 7.91, 9.52 and 5.00 on the CUB, Oxford and COCO datasets. The code and data will be available on GitHub (https://github.com/senmaoy/RAT-Diffusion).","url_abs":"https://arxiv.org/abs/2410.01638v1","url_pdf":"https://arxiv.org/pdf/2410.01638v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"data-extrapolation-for-text-to-image","repo_url":"https://github.com/senmaoy/RAT-Diffusion","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"text-to-image-generation-1","task_name":"Text to Image Generation"},{"task_slug":"text-to-image-generation","task_name":"Text-to-Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/text-to-image-generation-on-coco","task":"Text-to-Image Generation","dataset":"COCO (Common Objects in Context)","model":"RAT-Diffusion","rank_in_archive_order":1,"of":69,"metrics":{"FID":"5.00"},"uses_additional_data":true},{"leaderboard":"/sota/text-to-image-generation-on-cub","task":"Text-to-Image Generation","dataset":"CUB","model":"RAT-Diffusion","rank_in_archive_order":1,"of":20,"metrics":{"FID":"6.36","Inception score":"6.56"},"uses_additional_data":true},{"leaderboard":"/sota/text-to-image-generation-on-oxford-102","task":"Text-to-Image Generation","dataset":"Oxford 102 Flowers","model":"RAT-Diffusion","rank_in_archive_order":1,"of":8,"metrics":{"FID":"9.52","Inception score":"4.35"},"uses_additional_data":true}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}