Stochastic Gradient Learning in Neural Networks

By eon BottouAT

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Li Deng
2. Yoshua Bengio
3. Dong Yu
4. Quoc V. Le
5. Bo Li
6. Mohit Bansal
7. Xu Tan
8. Tara N. Sainath

Citations in the corpus

• Multilingual acoustic models using distributed deep neural networks (2013, methodology): “Downpour SGD is more robust to machines failures than standard (synchronous) SGD because if one machine in a model replica fails, the other model replicas continue processing their training data and updating the model parameters via the parameter servers.”
• Convolutional Neural Networks-based continuous speech recognition using raw speech signal (2015, methodology): “Maximizing this likelihood is performed using the stochastic gradient ascent algorithm [19].”
• Rectified linear neural networks with tied-scalar regularization for LVCSR (2015, methodology): “Therefore, the stochastic gradient descent (SGD) algorithm [9, 10, 11] is usually used to optimize the non-convex objective function to learn DNNs, which is believed to help the learning to escape from poor local optima.”
• Extracting bottleneck features and word-like pairs from untranscribed speech for feature representation (2017, methodology): “The DNN took 40-dimensional BNFs as input, and it was trained by minibatch stochastic gradient descent (SGD) [22].”