Learning Convolutional Feature Hierarchies for Visual Recognition

By Koray Kavukcuoglu, Pierre Sermanet, Y-Lan Boureau, Karol Gregor, Michaël Mathieu, Yann LeCun. 2010

594 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Towards Conceptual Compression (2016, methodology): “Numerous techniques exist for unsupervised learning in deep networks, e.g. sparse auto-encoders and sparse coding (Kavukcuoglu et al., 2010; Le, 2013), denoising auto-encoders (Vincent et al., 2010), deconvolutional networks (Zeiler et al., 2010), restricted Boltzmann machines (Hin-ton &…”
Pedestrian Detection with Unsupervised Multi-stage Feature Learning (2012, methodology): “Recent works have demonstrated the usefulness of unsupervised pre-training for end-to-end training of deep multi-stage architectures using a variety of techniques such as stacked restricted Boltzmann machines [16], stacked auto-encoders [4] and stacked sparse auto-encoders [32], and using new types ”
Speeding up Convolutional Neural Networks with Low Rank Expansions (2014, background): “The collection of filters F can be learnt, for example, through dictionary learning methods [18, 20, 31] or CNNs, and are generally full rank and expensive to convolve with large images.”
Representation Learning: A Review and New Perspectives (2012, methodology): “The main idea can be summarized by the following equation for the training criterion, which is simultaneously optimized with respect to hidden codes (representation) h(t) and with respect to parameters (W,α): JPSD = ∑ t λ‖h(t)‖1+‖x(t)−Wh(t)‖22+‖h(t)− fα(x(t))‖22 (26) where x(t) is the input vector f”
Energy-based Generative Adversarial Network (2016, methodology): “ore than an identity function. By adding regularizers, several techniques were addressed to resolve such problem (Vincent et al., 2010; Rifai et al., 2011; Ng; MarcAurelio Ranzato & Chopra, 2007; Kavukcuoglu et al., 2010). However, we claim that the EBGAN framework is free from such concern. On ”
Stacked What-Where Auto-encoders (2015, background): “This “what”/“where” concept also links to invariant feature hierarchies learning schemes [4, 5, 8, 9, 10, 21, 22].”
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation (2015, background): “[47] to accept full image sizes as input to learn hierarchical encoders.”
Deep Learning: Methods and Applications (2014, positive): “Deep models with convolution structure such as CNNs have been found effective and have been in use in computer vision and image recognition since 90s [57, 185, 192, 198, 212].”
Learning Invariant Representations with Local Transformations (2012, methodology): “0 % These results suggest that TIRBMs can learn diverse sets of ﬁlters, which is reminiscent of the eﬀects of convolutional training (Kavukcuoglu et al., 2010).”
Large-Scale Feature Learning With Spike-and-Slab Sparse Coding (2012, methodology): “, 2007), and hybrids of autoencoders and sparse coding (Kavukcuoglu et al., 2010).”
A tutorial survey of architectures, algorithms, and applications for deep learning (2014, background): “Deep architectures with convolution structure have been found highly effective and been commonly used in computer vision and image recognition (Bengio and LeCun, 1995; LeCun et al., 1998; Jarrett et al., 2009; Kavukcuoglu et al., 2010; Ciresan et al., 2012; Le et al., 2012; Dean et al., 2012; Krizhe”
Learning to Label Aerial Images from Noisy Data (2012, background): “Our general architecture has some similarity to various convolutional architectures that have recently become popular (Lee et al., 2009; Kavukcuoglu et al., 2010).”
Encoding Time Series as Images for Visual Inspection and Classification Using Tiled Convolutional Neural Networks (2014, background): “2010), sparse coding and Topographic Independent Component Analysis (TICA) are integrated as unsupervised pretraining approaches to learn more diverse features with complex invariances (Kavukcuoglu et al. 2010; Ngiam et al. 2010).”
Moving Beyond Feature Design: Deep Architectures and Automatic Feature Learning in Music Informatics (2012, background): “Additionally, auto-encoder architectures are a recent addition to the unsupervised training landscape and offer similar potential [21].”
Vehicle Type Classification Using a Semisupervised Convolutional Neural Network (2015, background): “The representations can thus be robust to geometric distortions and small shifts.”
Depth Map Super-Resolution by Deep Multi-Scale Guidance (2016, methodology): “With the motivation from convolutional dictionaries [21], Osendorfer et al.”
Sparse Modeling for Image and Vision Processing (2014, methodology): “It has been revisited recently, and has been shown useful for various recognition tasks [Zeiler et al., 2010, 2011, Rigamonti et al., 2011, 2013, Kavukcuoglu et al., 2010a] Specifically, let us consider an image x represented by a vector of size l and let the binary matrix Rk in {0, 1}m×l be the…”
Unsupervised learning of visual invariance with temporal coherence (2011, background): “Table 2: Results on Caltech 101 dataset Two-layer ConvNet [31] 66.”
A Recursive Recurrent Neural Network for Statistical Machine Translation (2014, methodology): “With the efﬁcient training methods, such as (Hinton et al., 2006), DNN is widely applied to speech and image processing, and has achieved breakthrough re-sults (Kavukcuoglu et al., 2010; Krizhevsky et al., 2012; Dahl et al., 2012).”