Rethinking the Inception Architecture for Computer Vision
By Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, Zbigniew Wojna. 2015
31,459 citations recorded by the source corpus.
Convolutional networks are at the core of most state of-the-art computer vision solutions for a wide variety of tasks. Since 2014 very deep convolutional networks started to become mainstream, yielding substantial gains in various benchmarks. Although increased model size and computational cost tend to translate to immediate quality gains for most tasks (as long as enough labeled data is provided for training), computational efficiency and low parameter count are still enabling factors for various use cases such as mobile vision and big-data scenarios. Here we are exploring ways to scale up networks in ways that aim at utilizing the added computation as efficiently as possible by suitably factorized convolutions and aggressive regularization. We benchmark our methods on the ILSVRC 2012 classification challenge validation set demonstrate substantial gains over the state of the art: 21:2% top-1 and 5:6% top-5 error for single frame evaluation using a network with a computational cost of 5 billion multiply-adds per inference and with using less than 25 million parameters. With an ensemble of 4 models and multi-crop evaluation, we report 3:5% top-5 error and 17:3% top-1 error on the validation set and 3:6% top-5 error on the official test set.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Quoc V. Le
- Yoshua Bengio
- Jianfeng Gao
- Percy Liang
- Sergey Levine
- Luke S. Zettlemoyer
- Dacheng Tao
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Trevor Darrell
- Xiaogang Wang
- Yi Yang
- Jitendra Malik
- Andrew Zisserman
In Model Architectures
- Quoc V. Le
- Yoshua Bengio
- Alan L. Yuille
- Jiashi Feng
- Jianfeng Gao
- Kaiming He
- Dacheng Tao
- Xiaogang Wang
Citations in the corpus
- Xception: Deep Learning with Depthwise Separable Convolutions (2016, methodology): “in 2014 [20] as GoogLeNet (Inception V1), later refined as Inception V2 [7], Inception V3 [21], and most recently Inception-ResNet [19].”
- Semi-Supervised Sequence Modeling with Cross-View Training (2018, methodology): “We apply label smoothing (Szegedy et al., 2016; Pereyra et al., 2017) with a rate of 0.1 to the target labels when training on the labeled data.”
- Masked Autoencoders Are Scalable Vision Learners (2021, neutral): “[52] 0.1 mixup [69] 0.8 cutmix [68] 1.0 drop path [30] 0.1 (B/L) 0.2 (H) Partial fine-tuning.”
- Matching Networks for One Shot Learning (2016, methodology): “The Inception Oracle (trained on all classes) performs almost perfectly when restricted to 5 classes only, which is not too surprising given its impressive top-1 accuracy.”
- Unified Language Model Pre-training for Natural Language Understanding and Generation (2019, methodology): “We also use label smoothing (Szegedy et al., 2016) with rate of 0 .”
- Measuring the Effects of Data Parallelism on Neural Network Training (2018, methodology): “They demonstrated speedups in both wall time and steps to convergence for an Inception model (Szegedy et al., 2016) on ImageNet (Russakovsky et al.”
- Do Better ImageNet Models Transfer Better? (2018, methodology): “[8] found that, out of three networks, the two more computationally expensive networks performed better on PASCAL VOC.”
- Self-Attention with Relative Position Representations (2018, neutral): “1 (Szegedy et al., 2016).”
- Scaling Neural Machine Translation (2018, background): “1 weight for the uniform prior distri-2 numrefs.1+smooth.exp+test.wmt14/full+tok.13a+ version.1.2.9 bution over the vocabulary (Szegedy et al., 2015; Pereyra et al., 2017).”
- Mixed Precision Training (2017, methodology): “2012) VGG-D 65.40% 65.43% (Simonyan and Zisserman, 2014) GoogLeNet (Inception v1) 68.33% 68.43% (Szegedy et al., 2015) Inception v2 70.03% 70.02% (Ioffe and Szegedy, 2015) Inception v3 73.85% 74.13% (Szegedy et al., 2016) Resnet50 75.92% 76.04% (He et al., 2016b) Loss-scaling technique was not requi”
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (2019, methodology): “Starting from 224x224 in early ConvNets, modern ConvNets tend to use 299x299 (Szegedy et al., 2016) or 331x331 (Zoph et al.”
- An Investigation into Neural Net Optimization via Hessian Eigenvalue Density (2019, background): “Scaling it to run on a 27 million parameter Inception V3 [27] on ImageNet (where we performed our largest scale experiments) requires a significant engineering effort.”
- Beyond English-Centric Multilingual Machine Translation (2020, methodology): “We train with the Adam optimizer (Kingma and Ba, 2015) and warmup first for 4000 updates, with label smoothing 0.1 (Szegedy et al., 2015; Pereyra et al., 2017).”
- Few-Shot Text Generation with Natural Language Instructions (2020, methodology): “1 (Szegedy et al., 2016); we also adopt Zhang et al. (2020)’s maximum output lengths for each task.”
- Autoregressive Entity Retrieval (2020, methodology): “, 2014) and label smoothing (Szegedy et al., 2016).”
- Are we done with ImageNet? (2020, methodology): “We start by assembling a set of 19 models [3, 27, 4, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37] which vary along a number of dimensions, including architecture, training objective and usage of external data (see appendix A).”
- Exploring Self-Attention for Image Recognition (2020, methodology): “We use label smoothing regularization with coefficient 0.1 [32].”