Auto-association by multilayer perceptrons and singular value decomposition

By Hervé Bourlard, Yves G. Kamp. 1988

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

Pylearn2: a machine learning research library (2013, methodology): “This includes autoencoders[6], RBMs [47] including RBMs with Gaussian visible units [54], DBMs [45], MLPs [44], convolutional networks [33], and local coordinate coding [56].”
What makes ImageNet good for transfer learning? (2016, background): “The most well-known such criteria are image reconstruction [7, 38, 31, 29, 34, 23] (see [6] for a comprehensive overview) and feature slowness [48, 18].”
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles (2016, background): “ctable when multiple layers are present and are not designed to produce features in an ecient manner. The direct mapping approach focuses on the latter aspect and is typically built via autoencoders [6,19,29]. Autoencoders specify explicitly the feature extraction function (encoder) in a parametric ”
From Pixels to Torques: Policy Learning with Deep Dynamical Models (2015, methodology): “The auto-encoder has strong similarities with principal component analysis (PCA): With a linear activation function the auto-encoder and the PCA are identical (Bourlard & Kamp, 1988).”
Representation Learning: A Review and New Perspectives (2012, methodology): “This is also true when using a sigmoid nonlinearity in the encoder [37], but not if the weights W and W 0 are tied (W 0 1⁄4W ) because W cannot be forced into being small and W 0 large to achieve a linear encoder.”
Learning to See by Moving (2015, background): “This hypothesis inspired research in fully unsupervised learning from visual imagery and led to popular algorithms such as ICA [5, 20], autoen-coders [24, 9], sparse coding [32], restricted boltzmann machines [27, 37] and others [6].”
UvA-DARE (Digital Academic Repository) Self-Supervised Video Representation Learning With Odd-One-Out Networks (positive): “The most common techniques studied include autoencoders [6, 20], restricted Boltzmann machines [19], convolutional deep belief networks [28], LSTMs and recurrent neural networks [36].”
A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines (2018, background): “All AEs thus share a common origin, which may be called the basic AE [53].”
Latent Space Physics: Towards Learning the Temporal Evolution of Fluid Flow (2018, positive): “tion of the input data. The inherent non-linearity is a particularly important aspect here. Previous works have thoroughly demonstrated the advantages over linear bases such as those computed by PCA [Bourlard and Kamp 1988], and the neural network latent space as basis for our algorithm is one of th”
Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, methodology): “Unsupervised learning is implemented in neural networks with self-organizing maps (SOM) (Kohonen, 1982, 1990), auto-encoders (Bourlard and Kamp, 1988), restricted Boltzmann machines (RBM) (Hinton and Salakhutdinov, 2006), Hebbian plasticity (Hebb, 1949; Gerstner and Kistler, 2002a; Cooper, 2005), an”
Encoder Based Lifelong Learning (2017, methodology): “An autoencoder [3] trained on the representation of the first task data obtained from an optimized model can be used to capture the most important features for this task.”
Expert Gate: Lifelong Learning with a Network of Experts (2016, background): “An autoencoder [4] is a neural network that learns to produce an output similar to its input [11].”
Domain Generalization for Object Recognition with Multi-task Autoencoders (2015, background): “To do so, we extend the autoencoder [7] into a model that jointly learns multiple data-reconstruction tasks taken from related domains.”
Learning deep dynamical models from image pixels (2014, methodology): “More precisely, if we use a linear activation function and only consider a single layer, the auto-encoder and PCA are identical [2].”
Deep Neural Networks for Learning Graph Representations (2016, methodology): “To train a deep architecture that involves multiple layers of feature representations, autoencoders have emerged as one of the commonly used building blocks (Bourlard and Kamp 1988; Hinton and Zemel 1994).”
Learning Deep Representations for Graph Clustering (2014, methodology): “In the greedy layerwised pretraining process, autoencoder (Bourlard and Kamp 1988) (Hinton and Zemel 1994) is commonly used as a basic unit to generate new representations in each layer, and it is also the main building block in our model.”
Learning to Schedule Communication in Multi-agent Reinforcement Learning (2019, methodology): “Impact of joint scheduling and encoding To study the effect of jointly coupling scheduling and encoding, we devise a comparison against a pre-trained auto-encoder (Bourlard & Kamp, 1988; Hinton & Zemel, 1994).”
Neural Machine Translation with Reconstruction (2016, background): “Reconstruction is a standard concept in auto-encoder, which is usually realized by a feed forward network (Bourlard and Kamp 1988; Vincent et al. 2010; Socher et al. 2011).”