Going deeper with convolutions
By Christian Szegedy, Wei David Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich. 2014
47,651 citations recorded by the source corpus.
We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yoshua Bengio
- Mohit Bansal
- Quoc V. Le
- Jianfeng Gao
- Sergey Levine
- Dacheng Tao
- Percy Liang
- Trevor Darrell
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Trevor Darrell
- Jitendra Malik
- Yi Yang
- Andrew Zisserman
In Model Architectures
- Yoshua Bengio
- Quoc V. Le
- Alan L. Yuille
- Dacheng Tao
- Jianfeng Gao
- Ruslan Salakhutdinov
- Kaiming He
- Jiashi Feng
Citations in the corpus
- Deep Residual Learning for Image Recognition (2015, background): “In [43, 24], a few intermediate layers are directly connected to auxiliary classifiers for addressing vanishing/exploding gradients.”
- Massive Exploration of Neural Machine Translation Architectures (2017, methodology): “We expected deep models to perform better (Zhou et al., 2016; Szegedy et al., 2015) across the board, and we believe that our experiments demonstrate the need for more robust techniques for optimizing deep sequential models.”
- Xception: Deep Learning with Depthwise Separable Convolutions (2016, methodology): “in 2014 [20] as GoogLeNet (Inception V1), later refined as Inception V2 [7], Inception V3 [21], and most recently Inception-ResNet [19].”
- Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation (2016, background): “In the past two years the top positions of the ImageNet contest are always occupied by the systems with more than tens or even hundreds of layers (Szegedy et al., 2015; He et al., 2015).”
- Rethinking the Inception Architecture for Computer Vision (2015, methodology): “VGGNet [18] and GoogLeNet [20] yielded similarly high performance in the 2014 ILSVRC [16] classification challenge.”
- Learning Distributed Representations of Sentences from Unlabelled Data (2016, methodology): “The image representations were encoded by a deep convolutional network (Szegedy et al., 2014) have identical architecture to SkipThought, but are trained on sentence-aligned translated texts.”
- Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books (2015, methodology): “We used the GoogLeNet architecture [36] as well as hybrid-CNN [42] for extracting frame features.”
- Training Very Deep Networks (2015, background): “Skip connections between layers or to output layers (where error is “injected”) have long been used in neural networks, more recently with the explicit aim to improve the flow of information [22, 23, 2, 24].”
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015, neutral): “1× 10(6) parameters (1536 × (4 + 1) × 800 for GoogleNet [34] in MultiBox [27]).”
- Object Detection via a Multi-region and Semantic Segmentation-Aware CNN Model (2015, neutral): “Besides object representation, our work is also motivated by the observation that, due to the remarkable classification capability of the recent CNN models [17, 30, 25, 16, 13, 26], the bottleneck for good detection performance is now the accurate object localization.”
- Highway Networks (2015, background): “…schemes (Glorot & Ben-gio, 2010; Saxe et al., 2013; He et al., 2015), techniques of training networks in multiple stages (Simonyan & Zis-serman, 2014; Romero et al., 2014) or with temporary companion loss functions attached to some of the layers (Szegedy et al., 2014; Lee et al., 2015).”
- Object Detection Networks on Convolutional Feature Maps (2015, methodology): “We compare network architectures of VGG-16 [15], GoogleNet [18], and ResNet-101 [17].”
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (2015, methodology): “The main difference to the network described in (Szegedy et al., 2014) is that the 5 × 5 convolutional layers are replaced by two consecutive layers of 3 × 3 convolutions with up to 128 filters. contains The network 13 .”
- FitNets: Hints for Thin Deep Nets (2014, background): “…are exponentially more expressive than shallow ones for some families of functions (Montufar et al., 2014); and (2) empirically: the two top-performers of ImageNet use deep convolutional networks with 19 and 22 layers, respectively (Simonyan & Zisserman, 2014) and (Szegedy et al., 2014).”
- Convolutional neural networks at constrained time cost (2014, methodology): “The recent models [1, 9, 22, 23] take a high-end GPU or multiple GPUs/clusters one week or several weeks to train, which can sometimes be too demanding for the rapidly changing industry.”
- Deeply-Supervised Nets (2014, methodology): “The result on the ImageNet classification task is also very encouraging (the most recent winning systems [26, 23] for this challenge involved many specific engineering steps and used a cluster of machines for training).”
- Very Deep Convolutional Networks for Large-Scale Image Recognition (2014, methodology): “op-5 test error (%) VGG (2 nets, multi-crop & dense eval.) 23.7 6.8 6.8 VGG (1 net, multi-crop & dense eval.) 24.4 7.1 7.0 VGG (ILSVRC submission, 7 nets, dense eval.) 24.7 7.5 7.3 GoogLeNet (Szegedy et al., 2014) (1 net) - 7.9 GoogLeNet (Szegedy et al., 2014) (7 nets) - 6.7 MSRA (He et al.,”
- ImageNet Large Scale Visual Recognition Challenge (2014, background): “9 Google Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Drago Anguelov, Dumitru Erhan, Andrew Rabinovich (Szegedy et al., 2014)”