What is the best multi-stage architecture for object recognition?

By Kevin Jarrett, Koray Kavukcuoglu, Marc'Aurelio Ranzato, Yann LeCun. 2009

2,391 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Generative Adversarial Nets (2014, methodology): “These striking successes have primarily been based on the backpropagation and dropout algorithms, using piecewise linear units [17, 8, 9] which have a particularly well-behaved gradient .”
Semi-Supervised Sequence Modeling with Cross-View Training (2018, background): “Early approaches to deep semi-supervised learning pre-train neural models on unlabeled data, which has been successful for applications in computer vision (Jarrett et al., 2009; LeCun et al., 2010) and NLP.”
How transferable are features in deep neural networks? (2014, methodology): “Whereas our networks have max pooling and local normalization on layers 1 and 2, just as Jarrett et al. (2009) did, we use a different nonlinearity (relu(x) instead of abs(tanh(x))), different layer sizes and number of layers, as well as other differences.”
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks (2013, background): “Recognizing the category of the dominant object in an image i s a tasks to which Convolutional Networks (ConvNets) [17] have been applied for many years, w hether the objects were handwritten characters [16], house numbers [24], textureless toys [18] , traffic signs [3, 26], objects from the Caltech”
Playing Atari with Deep Reinforcement Learning (2013, neutral): “The first hidden layer convolves 16 8× 8 filters with stride 4 with the input image and applies a rectifier nonlinearity [10, 18].”
Adversarial Training Methods for Semi-Supervised Text Classification (2016, background): “The activation function on the hidden layer was ReLU(Jarrett et al., 2009; Nair & Hinton, 2010; Glorot et al., 2011).”
Group Normalization (2018, methodology): “Normalization layers in deep networks had been widely used before the development of BN. Local Response Nor-malization (LRN) [40, 28, 32] was a component in AlexNet [32] and following models [64, 53, 58].”
Explaining and Harnessing Adversarial Examples (2014, background): “LSTMs (Hochreiter & Schmidhuber, 1997), ReLUs (Jarrett et al., 2009; Glorot et al., 2011), and maxout networks (Good-fellow et al., 2013c) are all intentionally designed to behave in very linear ways, so that they are easier to optimize.”
Grounded Compositional Semantics for Finding and Describing Images with Sentences (2014, methodology): “The layers consist of three sequences of filtering, pooling and local contrast normalization (Jarrett et al., 2009).”
Visualizing and Understanding Convolutional Networks (2013, background): “For more details of these operations, see (Krizhevsky et al., 2012) and (Jarrett et al., 2009).”
Conditional Generative Adversarial Nets (2014, methodology): “Both z and y are mapped to hidden layers with Rectified Linear Unit (ReLu) activation [4, 12], with layer sizes 200 and 1000 respectively, before both being mapped to second, combined hidden ReLu layer of dimensionality 1200.”
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition (2013, methodology): “Our method also outperforms by over 20% the two-layer convolutional network of Jarrett et al. (2009), demonstrating the importance of the depth of the network used for our feature.”
Generative adversarial networks (2014, positive): “ments We trained adversarial nets an a range of datasets including MNIST[23], the Toronto Face Database (TFD) [28], and CIFAR-10 [21]. The generator nets used a mixture of rectiﬁer linear activations [19, 9] and sigmoid activations, while the discriminator net used maxout [10] activations. Dropout [”
Representation Learning: A Review and New Perspectives (2012, background): “Structured sparsity penalties (Kavukcuoglu et al., 2009; Jenatton et al., 2009; Bach et al., 2011; Gregor et al., 2011) could be used for this purpose.”
AutoAugment: Learning Augmentation Policies from Data (2018, background): “Network architectures can also be used to hardcode invariances: convolutional networks bake in translation invariance [3–6], whereas physics models bake in invariance to translations, rotations, and permutations of atoms [7–12].”
Neural Architecture Search with Reinforcement Learning (2016, background): “Additionally, it is also possible to predict pooling, local contrast normalization (Jarrett et al., 2009; Krizhevsky et al., 2012), and batchnorm (Ioffe & Szegedy, 2015) in the architectures.”