Transformer in Convolutional Neural Networks

By Yun Liu, Guolei Sun, Yu Qiu, Le Zhang, Ajad Chhatkuli, Luc Van Gool. 2021

71 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Transformers in Vision: A Survey (2021, background): “TransCNN [114] propose a hierarchical multi-head self attention block, which first learns interactions within small grids (tokens) using self-attention, and then gradually merges the smaller grids into larger grids.”
LocalViT: Analyzing Locality in Vision Transformers (2021, unknown): “This aim of this paper is aligned with other works that try to answer this interesting question, i.e. taking the advantage of both convolution and transformers [12], [13], [14], [15], [16], [17].”
P2T: Pyramid Pooling Transformer for Scene Understanding (2021, methodology): “Following previous backbone architectures [4], [20], [21], [23], [25], we build P2T with different depths via stacking different number of pyramid pooling transformers at each stage.”
Self-Training Multi-Sequence Learning with Transformer for Weakly Supervised Video Anomaly Detection (2022, methodology): “Any action recognition method can be used as the Backbone, such as C3D (Tran et al. 2015), I3D (Carreira and Zisserman 2017), and VideoSwin (Liu et al. 2021c).”
SwinIR: Image Restoration Using Swin Transformer (2021, methodology): “When used in vision problems such as image classification [66, 19, 84, 56, 45, 55, 75], object detection [6, 53, 74, 56], segmentation [84, 99, 56, 4] and crowd counting [47, 69], it learns to attend to important image regions by exploring the global interactions between different regions.”
CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attention (2021, methodology): “…2021b), NesT (Zhang et al., 2021b), CvT (Wu et al., 2021), ViL (Zhang et al., 2021a), CAT (Lin et al., 2021), ResT (Zhang & Yang, 2021), TransCNN (Liu et al., 2021a), Shufﬂe (Huang et al., 2021), BoTNet (Srinivas et al., 2021), and RegionViT (Chen et al., 2021b We use MMDetection-based (Chen et…”
Efficient Long-Range Attention Network for Image Super-resolution (2022, background): “The SA mechanism in transformers can effectively model the dependency across data, and it has achieved impressive results on several high-level vision tasks, such as image classification [13, 26, 34, 35, 44, 52], image detection [8, 32, 35, 52], and segmentation [7, 35, 57, 63].”