{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generative-adversarial-transformers","title":"Generative Adversarial Transformers","arxiv_id":"2103.01209","date":"2021-03-01","proceeding":null,"authors":["Drew A. Hudson","C. Lawrence Zitnick"],"abstract":"We introduce the GANformer, a novel and efficient type of transformer, and explore it for the task of visual generative modeling. The network employs a bipartite structure that enables long-range interactions across the image, while maintaining computation of linear efficiency, that can readily scale to high-resolution synthesis. It iteratively propagates information from a set of latent variables to the evolving visual features and vice versa, to support the refinement of each in light of the other and encourage the emergence of compositional representations of objects and scenes. In contrast to the classic transformer architecture, it utilizes multiplicative integration that allows flexible region-based modulation, and can thus be seen as a generalization of the successful StyleGAN network. We demonstrate the model's strength and robustness through a careful evaluation over a range of datasets, from simulated multi-object environments to rich real-world indoor and outdoor scenes, showing it achieves state-of-the-art results in terms of image quality and diversity, while enjoying fast learning and better data-efficiency. Further qualitative and quantitative experiments offer us an insight into the model's inner workings, revealing improved interpretability and stronger disentanglement, and illustrating the benefits and efficacy of our approach. An implementation of the model is available at https://github.com/dorarad/gansformer.","url_abs":"https://arxiv.org/abs/2103.01209v4","url_pdf":"https://arxiv.org/pdf/2103.01209v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generative-adversarial-transformers","repo_url":"https://github.com/dorarad/gansformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"generative-adversarial-transformers","repo_url":"https://github.com/lucidrains/transganformer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"disentanglement","task_name":"Disentanglement"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"scene-generation","task_name":"Scene Generation"}],"methods":[{"method_slug":"adaptive-instance-normalization","method_name":"Adaptive Instance Normalization"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"feedforward-network","method_name":"Feedforward Network"},{"method_slug":"ganformer","method_name":"GANformer"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"r1-regularization","method_name":"R1 Regularization"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"stylegan","method_name":"StyleGAN"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-generation-on-clevr","task":"Image Generation","dataset":"CLEVR","model":"GANformer","rank_in_archive_order":2,"of":6,"metrics":{"FID-5k-training-steps":"9.1679"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-clevr","task":"Image Generation","dataset":"CLEVR","model":"StyleGAN2","rank_in_archive_order":3,"of":6,"metrics":{"FID-5k-training-steps":"16.0534"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-clevr","task":"Image Generation","dataset":"CLEVR","model":"GAN","rank_in_archive_order":4,"of":6,"metrics":{"FID-5k-training-steps":"25.0244"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-clevr","task":"Image Generation","dataset":"CLEVR","model":"SAGAN","rank_in_archive_order":5,"of":6,"metrics":{"FID-5k-training-steps":"26.0433"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-clevr","task":"Image Generation","dataset":"CLEVR","model":"VQGAN","rank_in_archive_order":6,"of":6,"metrics":{"FID-5k-training-steps":"32.6031"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-cityscapes","task":"Image Generation","dataset":"Cityscapes","model":"GANformer","rank_in_archive_order":2,"of":6,"metrics":{"FID-10k-training-steps":"5.7589"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-cityscapes","task":"Image Generation","dataset":"Cityscapes","model":"StyleGAN2","rank_in_archive_order":3,"of":6,"metrics":{"FID-10k-training-steps":"8.35"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-cityscapes","task":"Image Generation","dataset":"Cityscapes","model":"GAN","rank_in_archive_order":4,"of":6,"metrics":{"FID-10k-training-steps":"11.5652"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-cityscapes","task":"Image Generation","dataset":"Cityscapes","model":"SAGAN","rank_in_archive_order":5,"of":6,"metrics":{"FID-10k-training-steps":"12.8077"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-cityscapes","task":"Image Generation","dataset":"Cityscapes","model":"VQGAN","rank_in_archive_order":6,"of":6,"metrics":{"FID-10k-training-steps":"173.7971"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-ffhq","task":"Image Generation","dataset":"FFHQ","model":"StyleGAN2","rank_in_archive_order":7,"of":12,"metrics":{"Clean-FID (70k)":"2.98","FID-10k-training-steps":"10.8309"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-ffhq","task":"Image Generation","dataset":"FFHQ","model":"GANsformer","rank_in_archive_order":9,"of":12,"metrics":{"FID-10k-training-steps":"12.8478"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-ffhq","task":"Image Generation","dataset":"FFHQ","model":"GAN","rank_in_archive_order":10,"of":12,"metrics":{"FID-10k-training-steps":"13.1844"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-ffhq","task":"Image Generation","dataset":"FFHQ","model":"SAGAN","rank_in_archive_order":11,"of":12,"metrics":{"FID-10k-training-steps":"16.2069"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-ffhq","task":"Image Generation","dataset":"FFHQ","model":"VQGAN","rank_in_archive_order":12,"of":12,"metrics":{"FID-10k-training-steps":"63.1165"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-ffhq-256-x-256","task":"Image Generation","dataset":"FFHQ 256 x 256","model":"GANFormer","rank_in_archive_order":29,"of":51,"metrics":{"FID":"7.42"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-lsun-bedroom-256-x-256","task":"Image Generation","dataset":"LSUN Bedroom 256 x 256","model":"GANformer","rank_in_archive_order":28,"of":32,"metrics":{"FID-10k-training-steps":"6.5085"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-lsun-bedroom-256-x-256","task":"Image Generation","dataset":"LSUN Bedroom 256 x 256","model":"StyleGAN2","rank_in_archive_order":29,"of":32,"metrics":{"FID-10k-training-steps":"11.5255"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-lsun-bedroom-256-x-256","task":"Image Generation","dataset":"LSUN Bedroom 256 x 256","model":"GAN","rank_in_archive_order":30,"of":32,"metrics":{"FID-10k-training-steps":"12.1567"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-lsun-bedroom-256-x-256","task":"Image Generation","dataset":"LSUN Bedroom 256 x 256","model":"SAGAN","rank_in_archive_order":31,"of":32,"metrics":{"FID-10k-training-steps":"14.0595"},"uses_additional_data":false},{"leaderboard":"/sota/image-generation-on-lsun-bedroom-256-x-256","task":"Image Generation","dataset":"LSUN Bedroom 256 x 256","model":"VQGAN","rank_in_archive_order":32,"of":32,"metrics":{"FID-10k-training-steps":"59.6333"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2103.01209","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.01209"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/dorarad/gansformer","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lucidrains/transganformer","reach":null}],"summary":{"ran":3,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"listed":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6202bbbd7907d938","entry":"ChanNorm","repo":"lucidrains/transganformer","repo_kind":"listed","path":"transganformer/transganformer.py","file_url":"https://github.com/lucidrains/transganformer/blob/HEAD/transganformer/transganformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6202bbbd7907d938"}},{"code_sha256_prefix":"ba2f5a6aebc8fa85","entry":"FixedPositionalEmbedding","repo":"lucidrains/transganformer","repo_kind":"listed","path":"transganformer/transganformer.py","file_url":"https://github.com/lucidrains/transganformer/blob/HEAD/transganformer/transganformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ba2f5a6aebc8fa85"}},{"code_sha256_prefix":"f655a887963da4f4","entry":"RotaryEmbedding","repo":"lucidrains/transganformer","repo_kind":"listed","path":"transganformer/transganformer.py","file_url":"https://github.com/lucidrains/transganformer/blob/HEAD/transganformer/transganformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f655a887963da4f4"}},{"code_sha256_prefix":"ced642870818826e","entry":"get_sin_cos","repo":"lucidrains/transganformer","repo_kind":"listed","path":"transganformer/transganformer.py","file_url":"https://github.com/lucidrains/transganformer/blob/HEAD/transganformer/transganformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ced642870818826e"}},{"code_sha256_prefix":"1ac67506c6a0368b","entry":"rotate_every_two","repo":"lucidrains/transganformer","repo_kind":"listed","path":"transganformer/transganformer.py","file_url":"https://github.com/lucidrains/transganformer/blob/HEAD/transganformer/transganformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ac67506c6a0368b"}},{"code_sha256_prefix":"82e38a4ffe03866d","entry":"Attention","repo":"lucidrains/transganformer","repo_kind":"listed","path":"transganformer/transganformer.py","file_url":"https://github.com/lucidrains/transganformer/blob/HEAD/transganformer/transganformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82e38a4ffe03866d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}