Efficient Learning using Forward-Backward Splitting

By John C. Duchi, Yoram Singer. 2009

121 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Ad click prediction: a view from the trenches (2013, methodology): “More sophisticated approaches such as FOBOS and truncated gradient do succeed in introducing sparsity [11, 20].”
Malicious URL Detection using Machine Learning: A Survey (2017, background): “Here we discuss two categories of representation learning: feature selection, where the features are evaluated and selected based on their performance, and sparsity regularization, where the feature selection is implicitly done by incorporating it into the objective function.”
A survey of Algorithms and Analysis for Adaptive Online Learning (2014, methodology): “As discussed in McMahan (2011), it is precisely the explicit representation of the full α1:t‖x‖1 terms that lets Native FTRL produce much sparser solutions when compared with the composite-objective Mirror Descent update with L1 regularization (equivalent to the FOBOS algorithm of Duchi and Singer (”
Learning Games and Rademacher Observations Losses (2015, unknown): “= θ⊤Γθ Ridge ‖θ‖∞ .= maxk |θk| ℓ∞ ‖θ‖Φ .= maxM∈Sd(M|θ|)⊤ξ SLOPE (33) (Bach et al., 2011; Bogdan et al., 2015; Duchi & Singer, 2009;Su & Candès, 2015).”
Learning Discriminative Part Detectors for Image Classification and Cosegmentation (2013, unknown): “To make it tractable, we propose to use a stochastic algorithm in which a subset (six random samples) of training images are sampled to approximate the gradients / subgradients [13].”
Information-Theoretic Lower Bounds on the Oracle Complexity of Stochastic Convex Optimization (2010, background): “A large body of recent work has focused on developing optimization algorithms specifically tailored to high-dimensional problems [4, 5, 6].”
A Critical Review of Recurrent Neural Networks for Sequence Learning (2015, background): “A growing body of recent work [Carpenter, 2008, Langford et al., 2009, Singer and Duchi, 2009, Lipton and Elkan, 2015]) shows that given sparse inputs to a linear model with any standard regularizer, sparsity can be fully exploited even if the gradient is not sparse (owing to regularization).”
Dual Averaging Methods for Regularized Stochastic Learning and Online Optimization (2009, methodology): “In particul ar, a forwardbackward splitting method (FOBOS) is studied in [17] for sol ving the same problems we consider.”
Proximal Reinforcement Learning: A New Theory of Sequential Decision Making in Primal-Dual Spaces (2014, methodology): “is Forward-Backward Splitting (FOBOS) [55]”
Follow-the-Regularized-Leader and Mirror Descent: Equivalence Theorems and L1 Regularization (2011, methodology): “However, by storing the index t of the last round on which g t,i was nonzero, the L 1 part of the update can be made lazy [Duchi and Singer, 2009].”
On Regularizing Rademacher Observation Losses (2016, unknown): “= maxM∈Sd(M|θ|)>ξ SLOPE (20) [Bach et al., 2011, Bogdan et al., 2015, Duchi and Singer, 2009, Su and Candès, 2015].”
Distributed Training Strategies for the Structured Perceptron (2010, unknown): “, 2008) and coordinate descent algorithms (Duchi and Singer, 2009).”