Learning invariant features through topographic filter maps

By Koray Kavukcuoglu, Marc'Aurelio Ranzato, Rob Fergus, Yann LeCun. 2009

348 citations recorded by the source corpus.

Subfields: Aerospace Engineering, Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Stacked What-Where Auto-encoders (2015, background): “This “what”/“where” concept also links to invariant feature hierarchies learning schemes [4, 5, 8, 9, 10, 21, 22].”
Marginalized Denoising Autoencoders for Domain Adaptation (2012, background): “Various forms of au-toencoders have been developed in the deep learning literature (Rumelhart et al., 1986; Baldi & Hornik, 1989; Kavukcuoglu et al., 2009; Lee et al., 2009; Vincent et al., 2008; Rifai et al., 2011).”
Disentangling Factors of Variation via Generative Entangling (2012, methodology): “, 2010), invariant predictive sparse decomposition (IPSD) (Kavukcuoglu et al., 2009), as well as in Boltzmann machine-based approaches (Ranzato and Hinton, 2010; Courville et al.”
Unsupervised Feature Learning from Temporal Data (2015, methodology): “This apporach is most similar to the one taken by Kavukcuoglu et al [10] when optimizing group sparsity.”
Practical Recommendations for Gradient-Based Training of Deep Architectures (2012, background): “In sparse coding and sparse predictive coding (Kavukcuoglu et al. , 2009) the activations are directly optimized and actual zeros are the expected result of the optimization.”
What regularized auto-encoders learn from the data-generating distribution (2012, background): “In recent years, a number of unsupervised feature learning algorithms have been proposed that are based on minimizing some form of reconstruction error, such as auto-encoder and sparse coding variants (Olshausen and Field, 1997; Bengio et al., 2007; Ranzato et al., 2007; Jain and Seung, 2008; Ranzat”
Exploiting Cyclic Symmetry in Convolutional Neural Networks (2016, background): “The model of Kavukcuoglu et al. (2009) is able to learn local invariance to arbitrary transformations by grouping filters into overlapping neighbour-”
Convolutional neural networks applied to house numbers digit classification (2012, methodology): “Lp-pooling has been used previously in [6, 16] and a theoretical analysis of this method is described in [1].”
Emergence of Complex-Like Cells in a Temporal Product Network with Local Receptive Fields (2010, methodology): “The second section of the paper introduces the use of sparsity criteria operating on local pools of simple cells (Hyvarinen and Hoyer, 2001; Kavukcuoglu et al., 2009).”
Learning Invariant Representations with Local Transformations (2012, background): “As another related work, feature learning methods with topographic maps can also learn invariant representations (Hyv¨arinen et al., 2001; Kavukcuoglu et al., 2009).”
Learning Where to Attend with Deep Architectures for Image Tracking (2011, background): “…2006; Ranzato & Hinton, 2010; Welling et al., 2005; Swersky et al., 2011), but autoencoders (Vincent et al., 2008), sparse coding (Olshausen & Field, 1996; Kavukcuoglu et al., 2009), two-layer ICA (K¨oster & Hyv¨arinen, 2007) and convolutional architectures (Lee et al., 2009) could also be adopted.”
Unsupervised Learning of Spatiotemporally Coherent Metrics (2014, methodology): “This approach is most similar to the one taken by Kavukcuoglu et al. (2009) when optimizing group sparsity.”
Optimization with Sparsity-Inducing Penalties (2011, background): “Sparsity-inducing regularizations built upon such group structures have resulted in good performances for background subtraction [62, 86, 88], topographic dictionary learning [72, 88], wavelet-based denoising [111], and for face recognition with corruption by occlusions [70].”
A Survey of Inductive Biases for Factorial Representation-Learning (2016, background): “A topographic map model incorporating sparsity can be found in [27].”
Learning with Submodular Functions: A Convex Optimization Perspective (2011, background): “, 2009b) – Dictionary elements “organized” in a tree or a grid (Kavukcuoglu et al., 2009; Jenatton et al., 2010; Mairal et al., 2010) • Stability and identifiability – Optimization problem minw∈Rp L(y,Xw) + λ‖w‖1 is unstable – “Codes” w often used in later processing (Mairal et al.”
A Hybrid Collaborative Filtering Model with Deep Structure for Recommender Systems (2017, background): “Existing literatures have shown that multiple layers stacked together can generate rich representations in hidden layers, and therefore leads to better performance for various tasks (Rifai et al. 2011; Chen et al. 2012; Kavukcuoglu et al. 2009; Glorot, Bordes, and Bengio 2011).”
Normalizing the Normalizers: Comparing and Extending Network Normalization Schemes (2016, background): “Regularization on Activations: Sparsity or group sparsity regularizers on the activations have shown to be effective in the past (Roz, 2008; Kavukcuoglu et al., 2009) and several regularizers have been proposed that act directly on the neural activations.”