Stochastic Learning

By L. Eon Bottou. 2003

394 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

DeltaGrad: Rapid retraining of machine learning models (2020, background): “More recently a lot of work (see e.g., Bottou, 1998; 2003; Zhang, 2004; Bousquet & Bottou, 2008; Bottou, 2010; Bottou et al., 2018) focuses on empirical risk minimization.”
BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning (2020, methodology): “The random noise from sampling mini-batches of data in SGD-like algorithms and random initialization of the deep neural networks, combined with the fact that there is a wide variety of local minima solutions in high dimensional optimization problem (Ge et al., 2015; Kawaguchi, 2016; Wen et al., 2019”
FlexiFaCT: Scalable Flexible Factorization of Coupled Tensors on Hadoop (2014, background): “Solutions range from alternating least squares (ALS) [20], to stochastic gradient descent (SGD) [7].”
Topmoumoute Online Natural Gradient Algorithm (2007, background): “[3] reviews the different gradient descent techniques in the online setting and discusses their respective properties.”
Supercharging Recommender Systems using Taxonomies for Learning User Purchase Behavior (2012, methodology): “Despite this approximation, SGD has been shown to work very well in practice [11, 9, 10], often outperforming other methods including the standard gradient descent.”
Label Ranking Methods based on the Plackett-Luce Model (2010, methodology): “In our implementation, we use a standard stochastic gradient descent algorithm (Bottou, 2004) that, in terms of efficiency, compared quite favorably with other gradientbased methods.”
Semisupervised Neural Networks for Efficient Hyperspectral Image Classification (2010, methodology): “Unlike batch gradient descent, SGD processes one example at a time.”
Excess-Risk of Distributed Stochastic Learners (2013, background): “…algorithm would minimize the expected value of the quadratic loss [28] (also referred to as the “delta rule” [29]): QLMS(w,hk,i,yk(i)) , ρ 2 ‖w‖2 + (yk(i)− hTk,iw)2 (2) The expectation of the loss function over the distribution of the data is referred to as the risk function [30, p. 16]: Jk(w) ,…”
A novel Frank-Wolfe algorithm. Analysis and applications to large-scale SVM training (2013, neutral): “C”
Understanding Dropout (2013, background): “As such, and with the appropriately decreasing learning rates, it is almost surely convergent like other forms of stochastic gradient descent [11, 4, 5].”
Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks (2015, background): “The convergence of SGD has been established (Bottou 2004).”
Large-scale learning of word relatedness with constraints (2012, neutral): “Learning proceeds by stochastic gradient ascent [4, 22].”
Fields of Experts (2009, background): “This so-called stochastic gradient ascent procedure (Bottou, 2004) sped up learning considerably.”
DeepChrome: deep-learning for predicting gene expression from histone modifications (2016, positive): “Through these maps, we obtain intuitive outputs for understanding the combinatori l effects of histone modifications on gene regulation.”
Regression on Fixed-Rank Positive Semidefinite Matrices: A Riemannian Approach (2010, background): “Online learning algorithms [Bottou, 2004] consider possibly infinite sets of samples {(Xt, yt)}t≥1, received one at a time.”
The dropout learning algorithm (2014, background): “stochastic gradient descent in standard neural networks [38, 13, 14].”