This PDF file includes: Materials and Methods

By G. Fiquet, A. Auzende, J. Siebert, A. Corgne, Hélène Bureau, Haruka Ozawa, G. Garbarino. 2009

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In AI for Physical Sciences

Citations in the corpus

Pushing Stochastic Gradient towards Second-Order Methods -- Backpropagation Learning with Transformations in Nonlinearities (2013, background): “Learning deep neural networks has become a popular topic since the invention of unsupervised pretraining [4].”
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets (2016, methodology): “Early methods were based on stacked (often denoising) autoencoders or restricted Boltzmann machines [10–13].”
Learning to generate chairs with convolutional neural networks (2014, background): “Prominent examples from this line of work are restricted Boltzmann machines (RBMs) [13] and Deep Boltzmann Machines (DBMs) [26], as well as the plethora of models derived from them [12, 22, 20, 29, 23].”
Unsupervised Speech Representation Learning Using WaveNet Autoencoders (2019, methodology): “The deep learning revolution was triggered by the development of hierarchical representation learning algorithms, such as stacked Restricted Boltzman Machines [1] and Denoising Autoencoders [2].”
Representation Learning: A Review and New Perspectives (2012, background): “Another early approach was to stack RBMs or autoencoders into a deep autoencoder [92].”
Building high-level features using large scale unsupervised learning (2011, methodology): “We also compare our method against several other algorithms such as deep autoencoders (Hinton & Salakhutdinov, 2006; Bengio et al., 2007) and K-means (Coates et al., 2011).”
Learning to See by Moving (2015, background): “[18] proposed using intrinsic reward signals from the robots to learn a system for predicting traversability of paths while the robot explores its environment.”
Least Squares Generative Adversarial Networks (2016, background): “RBM [8], DBM [28] and VAE [14], have been proposed, these models face the difficulty of intractable functions or the difficulty of intractable inference, which in turn restricts the effectiveness of these models.”
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images (2014, neutral): “Error-rates for trained DNNi are similar to DNN1 (supplementary material).”
Progressive Neural Networks (2016, background): “Pretraining and finetuning was proposed in [7] and applied to transfer learning in [4, 11], generally in unsupervised-to-supervised or supervised-to-supervised settings.”
Learning to Generate Reviews and Discovering Sentiment (2017, background): “Much of the early research into modern deep learning was developed and validated via this approach (Hinton & Salakhutdinov, 2006) (Huang et al., 2007) (Vincent et al., 2008) (Coates et al., 2010) (Le, 2013).”
Representation Learning by Learning to Count (2017, methodology): “An example of this approach is the autoencoder [14, 40], which reconstructs data by mapping it to a low-dimensional feature vector.”
Multimodal Deep Learning (2011, methodology): “To motivate our deep autoencoder [5] model, we first describe several simple models and their drawbacks.”
Understanding deep image representations by inverting them (2014, background): “Finally, auto-encoder architectures [9] train networks together with their inverses as a form of supervision; here we are interested instead in visualising feedforward and discriminatively-trained CNNs now popular in computer vision.”
How to Construct Deep Recurrent Neural Networks (2013, methodology): “…prediction, character-level and word-level language modeling), the stacked RNN and the DOT(S)-RNN were initialized with the weights of the conventional RNN and the DT(S)-RNN, which is similar to layer-wise pretraining of a feedforward neural network (see, e.g., Hinton and Salakhutdinov, 2006).”
Training generative neural networks via Maximum Mean Discrepancy optimization (2015, methodology): “Some of the most successful approaches have been based on restricted Boltzmann machines [10] and deep Boltzmann networks [3].”
Stacked Generative Adversarial Networks (2016, methodology): “Some early efforts include Restricted Boltzmann Machines [22] and Deep Belief Networks [23].”
Optimizing Neural Networks with Kronecker-factored Approximate Curvature (2015, methodology): “…performance of K-FAC we applied it to the 3 deep-autoencoder optimization problems from Hinton and Salakhutdinov (2006), which use the “MNIST”, “CURVES”, and “FACES” datasets respectively (see Hinton and Salakhutdinov (2006) for a complete description of the network architectures and datasets).”