Transformation Invariance in Pattern Recognition-Tangent Distance and Tangent Propagation

By Patrice Y. Simard, Yann LeCun, John S. Denker, Bernard Victorri. 1996

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Unsupervised Data Augmentation for Consistency Training (2019, background): “In this work, we investigate the role of noise injection in consistency training and observe that advanced data augmentation methods, specifically those work best in supervised learning (Simard et al., 1998; Krizhevsky et al., 2012; Cubuk et al., 2018; Yu et al., 2018), also perform well in semisupe”
mixup: Beyond Empirical Risk Minimization (2017, methodology): “While data augmentation consistently leads to improved generalization (Simard et al., 1998), the procedure is dataset-dependent, and thus requires the use of expert knowledge.”
Temporal Ensembling for Semi-Supervised Learning (2016, background): “For further analysis about the importance of the gradient of the mapping function, see Simard et al. (1998).”
Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning (2016, background): “Tangent distance and tangent propagation proposed by [31] enforce local classification invariance with respect to the transformations of input images.”
Measuring and regularizing networks in function space (2018, background): “It appears that many approaches to normalize and whiten activations or gradients have been forwarded in the literature (Raiko et al. (2012);Simard et al. (1998); Schraudolph & Sejnowski (1996); Crammer et al. (2009); Wang et al. (2013); LeCun et al. (1991); Schraudolph (1998); Salimans & Kingma…”
Manitest: Are classifiers really invariant? (2015, methodology): “…in classiﬁers has been outlined in recent works [22, 30], and effective solutions for transformation-invariant classiﬁcations have been proposed by either adapting the classiﬁcation rules with proper distance metrics [11, 16, 29, 36], or by improving the features used for classiﬁcation [1, 4, 24].”
EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks (2019, methodology): “Automatic data augmentation is commonly used in vision (Simard et al., 1998; Szegedy et al., 2014; Krizhevsky et al., 2017) and speech (Cui et al., 2015; Ko et al., 2015) and can help train more robust models, particularly when using smaller datasets.”
Support vector machine classification with indefinite kernels (2007, background): “[31] or Haasdonk and Keysers [13], are invariant to various simple image transformations and have also shown excellent performance in optical character recognition.”
Unsupervised Data Augmentation (2019, methodology): “Although data augmentation has been studied extensively and has led to significant improvements, it has mostly been applied in supervised learning settings [Simard et al., 1998, Krizhevsky et al., 2012, Cubuk et al., 2018, Yu et al., 2018].”
Illumination Invariant Face Recognition Using Near-Infrared Images (2007, background): “Extrinsic factors, including eyeglasses, hairstyle, expression, posture, and environmental lighting, which make distributions of face data highly complex [5], [6], [7], should be minimized for reliable face recognition.”
Semi-supervised Learning with GANs: Manifold Invariance with Improved Inference (2017, methodology): “We use the tangent propagation approach (TangentProp) [36] to make the classifier invariant to the estimated tangent directions from the previous section.”
A Kernel Theory of Modern Data Augmentation (2018, background): “One illustrative example is that of tangent propagation for neural networks [36, 37], which proposes a regularization penalty to enforce local invariance, and has been extended in several recent works [32, 9, 43].”
Knowledge Transfer in Deep convolutional Neural Nets (2007, methodology): “In order to then incorporate this information, EBNN’s are trained using TangentProp (Simard et al. 2001).”
Image Quality Assessment: Unifying Structure and Texture Similarity (2020, methodology): “It generalizes many IQA models, including MSE, space/frequency weighting [18], [65], transform domain masking [20], and the tangent distance [66].”
What Happened to My Dog in That Network: Unraveling Top-down Generators in Convolutional Neural Networks (2015, unknown): “Another early reference touching on flows is (Simard et al., 1998); however, the flows here are computed from image pixels rather than from CNN features.”