Why Does Unsupervised Pre-training Help Deep Learning?

By P. Glöckner. 2013

318 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Research on exchange rate forecasting based on deep belief network (2017, unknown): “Erhan [12] explained the importance of unsupervised training.”
Investigation of transfer learning for ASR using LF-MMI trained neural networks (2017, methodology): “In an alternate paradigm, pretraining [6, 7] has been used to implicitly learn intermediate representations that are useful for different tasks.”
Modeling, Inference and Optimization With Composable Differentiable Procedures (2016, methodology): “Because learning schedules can implicitly regularize networks [28], for example by enforcing early stopping, for this experiment we optimized the learning rate schedules on the training error rather than on the validation set error.”
Estimating crop yields with deep learning and remotely sensed data (2015, background): “By performing pre-training, features of the input side can be efficiently propagated to the output side [8].”
Stock prediction using deep learning (2016, unknown): “As suggested by Zeiler [3] ADADELTA is used to overcome these limitations.”