ImageNet classification with deep convolutional neural networks

By Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton. 2012

130,310 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Residual Learning for Image Recognition (2015, methodology): “Our implementation for ImageNet follows the practice in [21, 40].”
Adam: A Method for Stochastic Optimization (2014, methodology): “SGD proved itself as an efficient and effective optimization method that was central in many machine learning success stories, such as recent advances in deep learning (Deng et al., 2013; Krizhevsky et al., 2012; Hinton & Salakhutdinov, 2006; Hinton et al., 2012a; Graves et al., 2013).”
Generative Adversarial Nets (2014, background): “So far, the most striking successes in deep learning have involved discriminative models, usually those that map a high-dimensional, rich sensory input to a class label [14, 20].”
Factorization tricks for LSTM networks (2017, methodology): “The partition of layer into parallel groups have been introduced by Krizhevsky et al. (2012) in AlexNet, where some convolutional layers have been divided into two groups to split the model between two GPUs.”
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer (2017, background): “This has been shown in domains such as text (Sutskever et al., 2014; Bahdanau et al., 2014; Jozefowicz et al., 2016; Wu et al., 2016), images (Krizhevsky et al., 2012; Le et al., 2012), and audio (Hinton et al., 2012; Amodei et al., 2015).”
Can Active Memory Replace Attention? (2016, background): “Recent successes of deep neural networks have spanned many domains, from computer vision [1] to speech recognition [2] and many other tasks.”
Xception: Deep Learning with Depthwise Separable Convolutions (2016, background): “In 2012, these ideas were refined into the AlexNet architecture [9], where convolution operations were being repeated multiple times in-between max-pooling operations, allowing the network to learn richer features at every spatial scale.”
Layer Normalization (2016, methodology): “Deep neural networks trained with some version of Stochastic Gradient Descent have been shown to substantially outperform previous approaches on various supervised learning tasks in computer vision [Krizhevsky et al., 2012] and speech processing [Hinton et al., 2012].”
Exploring the Limits of Language Modeling (2016, background): “, 2009), which enabled learning complex vision models from large amounts of data (Krizhevsky et al., 2012).”
Rethinking the Inception Architecture for Computer Vision (2015, methodology): “Since the 2012 ImageNet competition [16] winning entry by Krizhevsky et al [9], their network “AlexNet” has been successfully applied to a larger variety of computer vision tasks, for example to object-detection [5], segmentation [12], human pose estimation [22], video classification [8], object tra”
Sequence to Sequence Learning with Neural Networks (2014, background): “Deep Neural Networks (DNNs) are extremely powerful machine learning models that achieve excellent performance on difficult problems such as speech recognition [13, 7] and visual object recognition [19, 6, 21, 20].”
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference (2017, background): “artificial neural network techniques have made it possible to train general-purpose feature extractors that, with no or minimal retraining, can extract useful features for a variety of styles of data (Krizhevsky et al., 2012; Zeiler and Fergus, 2014; Donahue et al., 2014).”
Semi-supervised Sequence Learning (2015, methodology): “We use the same method as in [16] to perform data augmentation.”
How transferable are features in deep neural networks? (2014, methodology): “We followed Donahue et al. (2013) in making a few minor departures from Krizhevsky et al. (2012) when training the convnets in this study.”
Training Very Deep Networks (2015, background): “For instance, within just a few years, the top-5 image classification accuracy on the 1000-class ImageNet dataset has increased from ∼84% [1] to ∼95% [2, 3] using deeper networks with rather small receptive fields [4, 5].”
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015, methodology): “For the regression loss, we use L where R is the robust loss function (smooth L 1 ) deﬁned in [5].”
Object Detection via a Multi-region and Semantic Segmentation-Aware CNN Model (2015, neutral): “R-CNN [10] uses Alex Krizhevsky’s Net [17] to extract features from box proposals provided by selective”