Stochastic Spectral Descent for Restricted Boltzmann Machines

By David Edwin Carlson, Volkan Cevher, Lawrence Carin. 2015

76 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization (2019, background): “Notable examples include replacing the coordinates of the gradient with only their sign (Seide et al., 2014; Carlson et al., 2015; Bernstein et al., 2018, 2019; Karimireddy et al., 2019), quantizing the individual coordinates (Alistarh et al., 2017; Wen et al., 2017), and low-rank approximation of…”
Error Feedback Fixes SignSGD and other Gradient Compression Schemes (2019, background): “…performing updates only based on the sign of each coordinate of the gradient have recently gaining popularity for training deep learning models Seide et al. [2014], Carlson et al. [2015], Wen et al. [2017], Balles and Hennig [2018], Bernstein et al. [2018], Zaheer et al. [2018], Liu et al. [2018].”
RMSProp and equilibrated adaptive learning rates for non-convex optimization (2015, neutral): “ction we uncover an unsuspected link between RMSProp and equilibration. RMSProp is an adaptative learning rate method that has found much success in practice (Tieleman & Hinton,2012;Korjus et al.;Carlson et al., 2015).Tieleman & Hinton(2012) propose to normalize the gradients by an exponenti”
Differentially Private Mixture of Generative Neural Networks (2017, background): “ning schemes focus on convex optimization problems [4,13,52]. Also, training neural networks typically requires to optimize non-convex objective functions – as with Restricted Boltzmann Machine (RBM) [11] and Variational Autoencoder (VAE) [28] – which is usually done through the application of Stoch”