TransGAN: Two Transformers Can Make One Strong GAN

By Shiyu Chang, Zhangyang Wang. 2021

234 citations recorded by the source corpus.

Subfields: Generative Models, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Generative Models

In Model Architectures

Citations in the corpus

Transformers in Vision: A Survey (2021, methodology): “TransGAN [91] builds a strong GAN model, free of any convolution operation, with both generator and discriminator based upon the Transformer model [201].”
Pre-Trained Image Processing Transformer (2020, methodology): “[39] propose the TransGAN to generate images using Transformer.”
Score Matching Model for Unbounded Data Score (2021, methodology): “We downsize the dataset from 96× 96 to 48× 48 following the baselines [31, 29].”
A Survey of Transformers (2021, background): “, image classification [14, 33, 88], object detection [13, 88, 168, 172], image generation [61, 94] and video processing [3, 115].”
A Survey on Vision Transformer (2020, methodology): “[111] proposed TransGAN, which build GAN using the transformer architecture.”
CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation (2021, background): “Recently, Transformer has been considered as an alternative architecture, and has achieved competitive performance on many computer vision tasks, like image recognition [9, 17], semantic/instance segmentation [21, 27], object detection [2,29], low-level vision [3,14], and image generation [13].”
Efficient Training of Visual Transformers with Small Datasets (2021, background): “Visual Transformers (VTs) are progressively emerging architectures in computer vision as an alternative to standard Convolutional Neural Networks (CNNs), and they have already been applied to many tasks, such as image classification [17, 53, 61, 35, 58, 60, 33, 59], object detection [4, 66, 14], seg”
MAXIM: Multi-Axis MLP for Image Processing (2022, background): “It is worth mentioning that our model is able to generate high quality sharp images, which are visually comparable to the state-of-the-art generative models [35, 120].”
Aggregating Nested Transformers (2021, methodology): “We compare NesT-decoder to TransGAN [28], a very recent method that uses a full Transformer as the generator, as well as a convnet baseline following the widely-used architecture from [59] (its computationally expensive self-attention module is removed).”
StyTr2: Image Style Transfer with Transformers (2021, methodology): “However, the increment of network depth could cause the loss of feature resolution and fine details [20].”
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search (2021, background): “This exciting break-through in vision transformers led by ViT [20] and DETR [8], are achieving competitive performance on various vision tasks, such as image classiﬁcation [20, 66, 79, 14, 9], object detection [8, 90, 62], semantic segmentation [88], and others [26, 50, 33].”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, background): “learn long-range dependencies, GANs can be built with selfattention components [104], [234], [253], however, full quadratic attention does not scale well to high dimensional data.”
Conformer: Local Features Coupling Global Representations for Visual Recognition (2021, methodology): “Recently, the transformer architecture [42] has been introduced to visual tasks [16, 47, 41, 51, 8, 9, 3, 55, 28].”
FaceFormer: Speech-Driven 3D Facial Animation with Transformers (2021, methodology): “Besides, transformer-based models and the variants have also been proposed in object detection [7], semantic segmentation [64], image generation [30], etc.”
A-ViT: Adaptive Tokens for Efficient Vision Transformer (2021, background): “Recently, vision transformers have gained an increasing popularity in the vision community and they have been successfully applied to a broad range of vision applications, such as image classification [11, 16, 32, 43, 48, 55], object detection [3, 7, 39], image generation [20,21], and semantic segme”
Improved Transformer for High-Resolution GANs (2021, background): “Recent attempts [23, 32] build a pure Transformer-based GAN by a careful design of attention hyper-parameters as well as upsampling layers.”
Beyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks (2021, methodology): “Subsequently, transformer methods have been successfully applied to many visual tasks, including image classification [27, 12], object detection [28], lower-level vision [29], semantic segmentation [30], tracking [31], video instance segmentation [32], image generation [33], multimodal learning [34]”
Pros and Cons of GAN Evaluation Measures: New Developments (2021, background): “…(Goodfellow et al., 2014), autoregressive models such as PixelCNNs (Oord et al., 2016), variational autoencoders (VAEs) (Kingma and Welling, 2013), and recently transformer GANs (Hudson and Zitnick, 2021; Jiang et al., 2021) have been constantly improving the state of the art in image generation.”