Nonlinear Autoassociation Is Not Equivalent to PCA

By Nathalie Japkowicz, Stephen Jose Hanson, Mark A. Gluck. 2000

190 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistical and Nonlinear Physics

Citations in the corpus

Relational autoencoder for feature extraction (2017, background): “Later, with the involvement of non-linear activation functions, autoencoder becomes non-linear and is capable of learning more useful features [35] than linear feature extraction methods.”
AET vs. AED: Unsupervised Representation Learning by Auto-Encoding Transformations Rather Than Data (2019, background): “The use of auto-encoder architecture in learning representations in an unsupervised fashion has been extensively studied in literature [13, 14, 27].”
A systematic study of the class imbalance problem in convolutional neural networks (2017, methodology): “Then, the classiﬁcation of a new example is made based on a reconstruction error between the input and output patterns, e.g. absolute error, squared sum of errors, Euclidean or Mahalanobis distance [46, 47, 48].”
Materials that couple sensing, actuation, computation, and communication (2015, methodology): “The features extracted using this approach, when used with a nearest neighbor classifier resulted in improved accuracy compared with using statistical, spectral and PCA features.”
Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion (2010, background): “…a nonlinearity such as a sigmoid is used in the encoder, things become a little more complicated: obtaining the PCA subspace is a likely possibility (Bourlard and Kamp, 1988) since it is possible to stay in the linear regime of the sigmoid, but arguably not the only one (Japkowicz et al., 2000).”
Deep learning via semi-supervised embedding (2008, methodology): “If the autoencoder is linear it corresponds to PCA (Japkowicz et al., 2000) and hence also MDS, making a clear link to the embedding algorithms we discussed in Section 2.”
Autoencoder with recurrent neural networks for video forgery detection (2017, methodology): “In this case the autoencoder goes beyond PCA, capturing multi-modal aspects of the input distribution [29].”
An Introduction to Deep Learning (2011, unknown): “Another important fact is that an AA with a nonlinear fenc differs from a PCA as it is able to capture multimodal aspects of the input distribution [17].”
Learning Deep Architectures for AI (2007, methodology): “Some of the deep architectures discussed below (Deep Belief Nets and Stacked Auto-Encoders) exploit as component or monitoring device a particular type of neural network: the auto-encoder, also called autoassociator, or Diabolo network [25, 79, 90, 156, 172].”
Contractive Auto-Encoders: Explicit Invariance During Feature Extraction (2011, methodology): “It is fairly well understood what PCA or ICA do, but much remains to be done to understand the characteristics and theoretical advantages of the representations learned by a Restricted Boltzmann Machine (Hinton et al., 2006), an auto-encoder (Bengio et al., 2007), sparse coding (Olshausen and Field,”
Large-Scale Learning of Embeddings with Reconstruction Sampling (2011, background): “It has been shown that Auto-Encoders (especially deep ones) go beyond Principal Component Analysis (PCA) by capturing multi-modal interactions in the input distribution (Japkowicz et al., 2000; Hinton and Salakhutdinov, 2006).”
Auto-encoder based dimensionality reduction (2016, background): “Some researchers committed themselves to investigating several fundamental issues of neural networks with one or two hidden layers [8] [1] [22] [23] [3] [9] [17] [25] [21] [27].”