The Manifold Tangent Classifier

By Salah Rifai, Yann Dauphin, Pascal Vincent, Yoshua Bengio, Xavier Muller. 2011

285 citations recorded by the source corpus.

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Better Mixing via Deep Representations (2012, methodology): “A sampling algorithm was recently proposed for CAEs (Rifai et al. , 2012).”
A Generative Process for Contractive Auto-Encoders (2012, methodology): “Following Rifai et al. (2011b), we will be using a crossentropy loss:”
Discriminative Unsupervised Feature Learning with Convolutional Neural Networks (2014, neutral): “[21] for a recent discussion).”
Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks (2015, methodology): “Algorithm PI-MNIST test error (%) with n labeled examples n = 100 n = 1000 All MTC (Rifai et al., 2011) 12.”
Semi-supervised Learning with Deep Generative Models (2014, methodology): “Some of the best results currently are obtained by the manifold tangent classifier (MTC) (Rifai et al., 2011) and the AtlasRBF method (Pitelis et al.”
Representation Learning: A Review and New Perspectives (2012, methodology): “7.2.3 Contractive Auto-Encoders Contractive Auto-Encoders (CAE), proposed by Rifai et al. (2011a), follow up on Denoising Auto-Encoders (DAE) and share a similar motivation of learning robust representations.”
Semi-supervised Learning with Ladder Networks (2015, background): “Embedding (Weston et al. , 2012) 16.86 5.73 1.5 Transductive SVM (from Weston et al. , 2012) 16.81 5.38 1.40* MTC (Rifai et al. , 2011b) 12.03 3.64 0.81 Pseudo-label (Lee, 2013) 10.49 3.46 AtlasRBF (Pitelis et al. , 2014) 8.10 ( ± 0.95) 3.68 ( ± 0.12) 1.31 DGN (Kingma et al. , 2014) 3.33 ( 0.14) 2.4”
Stacked What-Where Auto-encoders (2015, background): “One widely accepted hypothesis about manifold [25] for classification states that samples from same class are likely to locate on the same sub-manifold whereas the ones from different classes lie across different sub-manifolds.”
Learning with Pseudo-Ensembles (2014, methodology): “From left-to-right the methods are Transductive SVM , neural net, convolutional neural net, EmbedNN [23], Manifold Tangent Classifier [16], Pseudo-Label [14], standard dropout plus fuzzing [9], dropout plus fuzzing with pre-training, PEA, and PEA with pre-training.”
Dropout Training as Adaptive Regularization (2013, background): “Dropout training falls into the broader category of learning methods that artificially corrupt training data to stabilize predictions [2, 4, 5, 6, 7].”
Practical Recommendations for Gradient-Based Training of Deep Architectures (2012, methodology): “In the CAE (Rifai et al., 2011a), this is achieved by adding an explicit regularizing term in the training criterion, proportional to the norm of the Jacobian of the encoder, ||∂f(x) ∂x ||2.”
What regularized auto-encoders learn from the data-generating distribution (2012, background): “This was first suggested in (Rifai et al., 2011c).”
Deep Bayesian Active Learning with Image Data (2017, neutral): “Embedding (Weston et al., 2012) 5.73% Transductive SVM (Weston et al., 2012) 5.38% MTC (Rifai et al., 2011) 3.64% Pseudo-label (Lee, 2013) 3.46% AtlasRBF (Pitelis et al., 2014) 3.68% DGN (Kingma et al., 2014) 2.40% Virtual Adversarial (Miyato et al., 2015) 1.32% Ladder Network ( Γ -model) (Rasmus…”
Deep Learning using Linear Support Vector Machines (2013, background): “These include (but not limited to) speech (Mohamed et al., 2009; Dahl et al., 2010) and vision (Jarrett et al., 2009; Ciresan et al., 2011; Rifai et al., 2011a; Krizhevsky et al., 2012).”
Deep Learning of Representations: Looking Forward (2013, methodology): “…priors on the form of the model can be used to enhance disentangling, such as multiplicative interactions between the factors (Tenenbaum and Freeman, 2000; Desjardins et al., 2012) or orthogonality of the features derivative with respect to the input (Rifai et al., 2011b, 2012a; Sohn et al., 2013).”
Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks (2013, background): “13 % achieved by the manifold tangent classifier having four hidden layers of logistic sigmoid units (Rifai et al., 2011).”
Smooth Neighbors on Teacher Graphs for Semi-Supervised Learning (2017, background): “MTC trains contrastive auto-encoders to learn the manifold and enforces the outputs to be insensitive to local perturbations along the low-dimensional manifold.”