Local Group Invariant Representations via Orbit Embeddings

By Anant Raj, Abhishek Kumar, Youssef Mroueh, Tom Fletcher, Bernhard Schölkopf. 2016

40 citations recorded by the source corpus.

Invariance to nuisance transformations is one of the desirable properties of effective representations. We consider transformations that form a \emph{group} and propose an approach based on kernel methods to derive local group invariant representations. Locality is achieved by defining a suitable probability distribution over the group which in turn induces distributions in the input feature space. We learn a decision function over these distributions by appealing to the powerful framework of kernel methods and generate local invariant random feature maps via kernel approximations. We show uniform convergence bounds for kernel approximation and provide excess risk bounds for learning with these features. We evaluate our method on three real datasets, including Rotated MNIST and CIFAR-10, and observe that it outperforms competing kernel based approaches. The proposed method also outperforms deep CNN on Rotated-MNIST and performs comparably to the recently proposed group-equivariant CNN.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network (2018, methodology): “Method RMSE MLP/Random CM [34] 5.96 LGIKA (RF) [35] 10.82 RBF Kernels/Random CM [34] 11.42 RBF Kernels/Sorted CM [34] 12.59 MLP/Sorted CM [34] 16.06 Spherical CNN [1] 8.47 Ours (FFS2CNN) 7.97 3D Shape Recognition Finally, we report results for shape classiﬁcation using the SHREC17 dataset [36],…”
Spherical CNNs (2018, neutral): “We evaluate by RMSE and compare our results to Montavon et al. (2012) and Raj et al. (2016) (see table 3).”
Variational Inference of Disentangled Latent Concepts from Unlabeled Observations (2017, background): “…knowledge about the nature of input transformations (e.g., rotations, translations, scaling for images, etc.) (Schmidt and Roth, 2012; Bruna and Mallat, 2013; Anselmi et al., 2014, 2016; Cohen and Welling, 2016; Dieleman et al., 2016; Haasdonk et al., 2005; Mroueh et al., 2015; Raj et al., 2017).”
Semi-supervised Learning with GANs: Manifold Invariance with Improved Inference (2017, methodology): “invariances and further improve on semi-supervised learning. This can be contrasted with methods that assume the knowledge of these invariances (e.g., rotation, translation, horizontal ﬂipping, etc.) [36, 18, 25, 31]. Estimating tangents for a real sample xrequires us to learn an encoder hthat maps ”
Invariance and Stability of Deep Convolutional Representations (2017, unknown): “The pooling operator on the group G is defined in a similar fashion as [22] by”
Group Invariance, Stability to Deformations, and Complexity of Deep Convolutional Representations (2017, unknown): “It is often useful to encode invariances to diﬀerent groups of transformations, such as rotations or reﬂections (see, e.g. , Cohen and Welling, 2016; Mallat, 2012; Raj et al., 2017; Sifre and Mallat, 2013).”
A Kernel Theory of Modern Data Augmentation (2018, methodology): “This feature map is exactly the embedding of the distribution of transformed points around x into the reproducing kernel Hilbert space (Muandet et al., 2017; Raj et al., 2017).”