{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/transgan-two-transformers-can-make-one-strong","title":"TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up","arxiv_id":"2102.07074","date":"2021-02-14","proceeding":"NeurIPS 2021 12","authors":["Yifan Jiang","Shiyu Chang","Zhangyang Wang"],"abstract":"The recent explosive interest on transformers has suggested their potential to become powerful \"universal\" models for computer vision tasks, such as classification, detection, and segmentation. While those attempts mainly study the discriminative models, we explore transformers on some more notoriously difficult vision tasks, e.g., generative adversarial networks (GANs). Our goal is to conduct the first pilot study in building a GAN completely free of convolutions, using only pure transformer-based architectures. Our vanilla GAN architecture, dubbed TransGAN, consists of a memory-friendly transformer-based generator that progressively increases feature resolution, and correspondingly a multi-scale discriminator to capture simultaneously semantic contexts and low-level textures. On top of them, we introduce the new module of grid self-attention for alleviating the memory bottleneck further, in order to scale up TransGAN to high-resolution generation. We also develop a unique training recipe including a series of techniques that can mitigate the training instability issues of TransGAN, such as data augmentation, modified normalization, and relative position encoding. Our best architecture achieves highly competitive performance compared to current state-of-the-art GANs using convolutional backbones. Specifically, TransGAN sets new state-of-the-art inception score of 10.43 and FID of 18.28 on STL-10, outperforming StyleGAN-V2. When it comes to higher-resolution (e.g. 256 x 256) generation tasks, such as on CelebA-HQ and LSUN-Church, TransGAN continues to produce diverse visual examples with high fidelity and impressive texture details. In addition, we dive deep into the transformer-based generation models to understand how their behaviors differ from convolutional ones, by visualizing training dynamics. The code is available at https://github.com/VITA-Group/TransGAN.","url_abs":"https://arxiv.org/abs/2102.07074v4","url_pdf":"https://arxiv.org/pdf/2102.07074v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"transgan-two-transformers-can-make-one-strong","repo_url":"https://github.com/VITA-Group/TransGAN","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"transgan-two-transformers-can-make-one-strong","repo_url":"https://github.com/0xzayd/transformer-GAN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"transgan-two-transformers-can-make-one-strong","repo_url":"https://github.com/asarigun/MixerGANsformer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"transgan-two-transformers-can-make-one-strong","repo_url":"https://github.com/asarigun/TransGAN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"transgan-two-transformers-can-make-one-strong","repo_url":"https://github.com/dsorge3/eeg_visual_classification_project","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"transgan-two-transformers-can-make-one-strong","repo_url":"https://github.com/hungtrankhanh/CS5260_project","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"transgan-two-transformers-can-make-one-strong","repo_url":"https://github.com/lucidrains/transganformer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"transgan-two-transformers-can-make-one-strong","repo_url":"https://github.com/milmor/TransGAN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"transgan-two-transformers-can-make-one-strong","repo_url":"https://github.com/omihub777/vit-cifar","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"transgan-two-transformers-can-make-one-strong","repo_url":"https://github.com/BR-IDL/PaddleViT/blob/main/gan/transGAN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"paddle","reach":null}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"image-generation","task_name":"Image Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-generation-on-celeba-64x64","task":"Image Generation","dataset":"CelebA 64x64","model":"TransGAN-XL","rank_in_archive_order":28,"of":39,"metrics":{"FID":"12.23"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-lsun-churches-256-x-256","task":"Image Generation","dataset":"LSUN Churches 256 x 256","model":"TransGAN","rank_in_archive_order":24,"of":27,"metrics":{"FID":"8.94"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-stl-10","task":"Image Generation","dataset":"STL-10","model":"TransGAN","rank_in_archive_order":14,"of":31,"metrics":{"FID":"18.28","Inception score":"10.43"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2102.07074","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2102.07074"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/omihub777/vit-cifar","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/asarigun/TransGAN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/asarigun/MixerGANsformer","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/milmor/TransGAN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/BR-IDL/PaddleViT/blob/main/gan/transGAN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lucidrains/transganformer","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/VITA-Group/TransGAN","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hungtrankhanh/CS5260_project","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dsorge3/eeg_visual_classification_project","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/0xzayd/transformer-GAN","reach":{"status":"unanswered"}}],"summary":{"ran_draft_wrong":4,"ran_fixture":2,"ran_violates":2},"by_repo_kind":{"listed":{"samples":5,"ran":5,"repositories":3}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"399c4e0c0bf77fb5","entry":"UpSampling","repo":"asarigun/TransGAN","repo_kind":"listed","path":"models.py","file_url":"https://github.com/asarigun/TransGAN/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"399c4e0c0bf77fb5"}},{"code_sha256_prefix":"7b15399f0ed06269","entry":"calculate_frechet_distance","repo":"dsorge3/eeg_visual_classification_project","repo_kind":"listed","path":"utils/fid_score.py","file_url":"https://github.com/dsorge3/eeg_visual_classification_project/blob/HEAD/utils/fid_score.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"7b15399f0ed06269"}},{"code_sha256_prefix":"85f67e8fb251251f","entry":"combine_contexts","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"85f67e8fb251251f"}},{"code_sha256_prefix":"60fff7c3c400d7ff","entry":"default","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"60fff7c3c400d7ff"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"6a9faa89f4d1a831","entry":"gelu","repo":"hungtrankhanh/CS5260_project","repo_kind":"listed","path":"models/TransGAN_8_8_1.py","file_url":"https://github.com/hungtrankhanh/CS5260_project/blob/HEAD/models/TransGAN_8_8_1.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"6a9faa89f4d1a831"}},{"code_sha256_prefix":"6b6b6f726f8ac029","entry":"get_attn_mask","repo":"hungtrankhanh/CS5260_project","repo_kind":"listed","path":"models/TransGAN_8_8_1.py","file_url":"https://github.com/hungtrankhanh/CS5260_project/blob/HEAD/models/TransGAN_8_8_1.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"6b6b6f726f8ac029"}},{"code_sha256_prefix":"9fadd038ce88c830","entry":"pixel_upsample","repo":"hungtrankhanh/CS5260_project","repo_kind":"listed","path":"models/TransGAN_8_8_1.py","file_url":"https://github.com/hungtrankhanh/CS5260_project/blob/HEAD/models/TransGAN_8_8_1.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"9fadd038ce88c830"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}