Xception: Deep Learning with Depthwise Separable Convolutions
By François Chollet. 2016
18,118 citations recorded by the source corpus.
We present an interpretation of Inception modules in convolutional neural networks as being an intermediate step in-between regular convolution and the depthwise separable convolution operation (a depthwise convolution followed by a pointwise convolution). In this light, a depthwise separable convolution can be understood as an Inception module with a maximally large number of towers. This observation leads us to propose a novel deep convolutional neural network architecture inspired by Inception, where Inception modules have been replaced with depthwise separable convolutions. We show that this architecture, dubbed Xception, slightly outperforms Inception V3 on the ImageNet dataset (which Inception V3 was designed for), and significantly outperforms Inception V3 on a larger image classification dataset comprising 350 million images and 17,000 classes. Since the Xception architecture has the same number of parameters as Inception V3, the performance gains are not due to increased capacity but rather to a more efficient use of model parameters.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Quoc V. Le
- Mohit Bansal
- Jianfeng Gao
- Yoshua Bengio
- Dacheng Tao
- Percy Liang
- Trevor Darrell
- Ruslan Salakhutdinov
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Xiaogang Wang
- Ming-Hsuan Yang
- Trevor Darrell
- Jitendra Malik
- Alan L. Yuille
- Yi Yang
In Model Architectures
- Quoc V. Le
- Alan L. Yuille
- Jiashi Feng
- Jianfeng Gao
- Shuicheng Yan
- Dacheng Tao
- Yoshua Bengio
- Christoph Feichtenhofer
Citations in the corpus
- Attention is All you Need (2017, background): “Separable convolutions [6], however, decrease the complexity considerably, to O(k · n · d + n · d(2)).”
- Factorization tricks for LSTM networks (2017, methodology): “This multi-group approach was extended to the extreme in Xception architecture by Chollet (2016).”
- QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension (2018, methodology): “We use depthwise separable convolutions (Chollet, 2016) (Kaiser et al., 2017) rather than traditional ones, as we observe that it is memory efficient and has better generalization.”
- Unsupervised Data Augmentation for Consistency Training (2019, methodology): “…of supervised examples: (a) We use 10% of the supervised data of ImageNet while using all other data as unlabeled data, (b) Secondly, we consider the fully supervised scenario where we keep all images in ImageNet as supervised data and obtain extra unlabeled data from the JFT dataset [25, 6].”
- Universal Transformers (2018, methodology): “Depending on the task, we use one of two different transition functions: either a separable convolution [5] or a fully-connected neural network that consists of a single rectified-linear activation function between two affine transformations, applied position-wise, i.”
- Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation (2020, methodology): “05 Xception-71 [21,16] 24.”
- Big Transfer (BiT): General Visual Representation Learning (2019, methodology): “ion images and 21k classes organized by the WordNet hierarchy. Images may contain multiple labels. BiT-L is trained on the JFT-300M dataset [51,38,61]. This dataset is a newer version of that used in [18,8]. JFT-300M consists of around 300 million images with 1.26 labels per image on average. The la”
- Stand-Alone Self-Attention in Vision Models (2019, background): “[53] also show improvements on image classification and achieve state-of-the-art results on video action recognition tasks with a variant of non-local means.”
- Local Relation Networks for Image Recognition (2019, methodology): “For greater efficiency, some methods consider only one or a group of input channels in producing one channel of the output feature [17, 5].”
- Group Normalization (2018, background): “2017) andXception (Chollet 2017) exploit channel-wise (also called “depth-wise”) convolutions, which are group convolutions with a group number equal to the channel number.”
- Relation Networks for Object Detection (2017, methodology): “The backbone network [42, 45, 43, 25, 8, 53] is pre-trained on ImageNet classification task [11] and fine-tuned during detection training.”
- Revisiting Unreasonable Effectiveness of Data in Deep Learning Era (2017, methodology): “We use the same mAP@100 metric as in [7], which is computed as the mean average precision (mAP) for top-100 predictions.”
- Measuring Robustness to Natural Distribution Shifts in Image Classification (2020, neutral): “[12] Chollet, F.”
- NVAE: A Deep Hierarchical Variational Autoencoder (2020, background): “The main building block of our network is depthwise convolutions [38, 39] that rapidly increase the receptive field of the network without dramatically increasing the number of parameters.”