Proximal regularization for online and batch learning

56 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Slow Learners are Fast (2009, methodology): “We extend this to bounds which are dependent on strong convexity [1, 7] to obtain adaptive algorithms which can take advantage of well-behaved optimization problems in practice.”
Proximal Algorithms (2013, background): “There is a wide literature on applying various proximal algorithms to particular problems or problem domains, such as nuclear norm problems [183], max norm problems [119], sparse inverse covariance selection [178], MAP inference in undirected graphical models [168], loss minimization in machine lear”
Follow-the-Regularized-Leader and Mirror Descent: Equivalence Theorems and L1 Regularization (2011, methodology): “If one uses R̃t(x) = σt 2 ‖x − xt‖ 2 2 in place of Rt(x), this algorithm reduces to standard online gradient descent [Do et al., 2009].”
On optimization methods for deep learning (2011, methodology): “Recent work on SGDs focuses on adaptive strategies for the learning rate (Shalev-Shwartz et al., 2007; Bartlett et al., 2008; Do et al., 2009) or improving SGD convergence by approximating second-order information (Vishwanathan et al.”