Unsupervised learning of invariant representations with low sample complexity: the magic of sensory cortex or a new framework for machine learning?

By Fabio Anselmi, Joel Z. Leibo, Lorenzo Rosasco, Jim Mutch, Andrea Tacchetti, Tomaso A. Poggio. 2013

86 citations recorded by the source corpus.

This work was supported by the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF - 1231216.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

Steerable CNNs (2016, methodology): “A theoretical account of equivariant representation learning in the brain is given by Anselmi et al. (2014). Group equivariant scattering networks were defined and studied by Mallat (2012) for compact groups, and by Sifre & Mallat (2013); Oyallon & Mallat (2015) for the roto-translation group. Jacob”
Group Equivariant Convolutional Networks (2016, background): “Anselmi et al. (2014; 2015) use the theory of locally compact topological groups to develop a theory of statistically efﬁcient learning in sensory cortex.”
Variational Inference of Disentangled Latent Concepts from Unlabeled Observations (2017, background): “…knowledge about the nature of input transformations (e.g., rotations, translations, scaling for images, etc.) (Schmidt and Roth, 2012; Bruna and Mallat, 2013; Anselmi et al., 2014, 2016; Cohen and Welling, 2016; Dieleman et al., 2016; Haasdonk et al., 2005; Mroueh et al., 2015; Raj et al., 2017).”
Memory-based Parameter Adaptation (2018, methodology): “In the context of learning invariant representations for object recognition, Anselmi et al. (2014) proposed a method that can invariantly and discriminatively represent objects using a single sample, even of a new class.”
Understanding deep convolutional networks (2016, background): “They are also studied as neuro-physiological models of vision [4].”
Transformation Properties of Learned Visual Representations (2014, background): “Much has been written about invariant representations (e.g. Anselmi et al. (2014)), and invariance to groups such as translations, rotations and projective transformations is indeed very important for object recognition.”
Learning from Conditional Distributions via Dual Embeddings (2016, background): “, 1998), speech recognition (Anselmi et al., 2013) and many other applications.”
Artificial General Intelligence: Concept, State of the Art, and Future Prospects (2009, background): “…under labels such as “deep learning” – e.g. Andrew Ng’s well known work applying deep learning to practical vision processing problems (Socher et al., 2012; Le, 2013), and the work of Tomasso Poggio and his team which achieves deep learning via simulations of visual cortex (Anselmi et al., 2013).”
Deep Symmetry Networks (2014, background): “[2] constructs features invariant to a symmetry group by using statistics of dot products with group orbits.”
The Invariance Hypothesis Implies Domain-Specific Regions in Visual Cortex (2014, methodology): “The templates used to implement invariance to group transformations need not be changed for different object classes while the templates implementing non-group invariance are class-specific.”
Why does Deep Learning work? - A perspective from Group Theory (2014, background): “Recently Anselmi et al. (2013) proposed a theory for visual-cortex that’s heavily inspired by the principles of group actions; although there is do direct connection with layer-wise pre-training in that context.”
Learning from Conditional Distributions via Dual Kernel Embeddings (2016, background): “We want to learn a function f which links the conditional distribution to target y ∈ Y: min f∈F L(f) = Ex,y [ `(y,Ez|x[f(z, x)]) ] (1) where ` : Y × R → R is a convex loss function, and f(z, x) = 〈f, ψ(z, x)〉 is a linear function in a suitably chosen (nonlinear) feature space.”
Generalization Error of Invariant Classifiers (2016, methodology): “Invariance of the CNN Architecture-Given that the set of transformations is a group, averaging a function over a group leads to an invariant representation (Anselmi et al., 2014; Bruna and Mallat, 2012; Co-hen and Welling, 2016).”
Unsupervised learning of invariant representations (2016, methodology): “The conjecture is proved, using a dimensionality reduction argument, for the special case of translation (and any Abelian group–see [6] for more details):”