Optimization with Sparsity-Inducing Penalties (Foundations and Trends(R) in Machine Learning)

By Francis R. Bach, Rodolphe Jenatton, Julien Mairal. 2011

331 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Learning to learn by gradient descent by gradient descent (2016, background): “In contrast, communities who focus on sparsity tend to favor very different approaches [Donoho, 2006, Bach et al., 2012].”
FastText.zip: Compressing text classification models (2016, background): “While many approaches have been proposed to select groups of variables during training (Bach et al., 2012; Meier et al., 2008), we are interested in selecting a ﬁxed subset of K words and ngrams from a pre-trained model.”
Frank-Wolfe Algorithms for Saddle Point Problems (2016, background): “…sparsity inducing norm (Martins et al., 2011) for the parameter x , such as the overlapping group lasso for which the projection is expensive (Bach et al., 2012), while Y could be a combinatorial object such as a the ground state of a planar Ising model (without external ﬁeld) which admits…”
Auto-Sizing Neural Networks: With Applications to n-gram Language Models (2015, background): “In fact, the two problems can be solved by nearly identical algorithms, because they are convex conjugates of each other (Duchi and Singer, 2009; Bach et al., 2012).”
Statistical physics of inference: thresholds and algorithms (2015, background): “This is particularly interesting since RBMs are the building blocks of “deep belief networks” [274] and have recently sparked a surge of interest, partly because of the efficient algorithms developed to train them (e.”
Primal-Dual Rates and Certificates (2016, background): “This includes group Lasso (L1/L2) and other norms inducing structured sparsity (Bach et al., 2012), as well as other penalties such as e.g. the fused Lasso g(α) := λ‖Mα‖1.”
Seismic sparse-spike deconvolution via Toeplitz-sparse matrix factorization (2016, methodology): “The proximity operator in equation 18 depends on the regularization functionΨð·Þ, which may not have an explicit form but can be efficiently computed in general (Bach et al., 2012).”
Why Does Deep and Cheap Learning Work So Well? (2016, background): “∗ Henry W. Lin, Max Tegmark, and David Rolnick Dept. of Physics, Harvard University, Cambridge, MA 02138 Dept. of Physics, Massachusetts Institute of Technology, Cambridge, MA 02139 and Dept. of Mathematics, Massachusetts Institute of Technology, Cambridge, MA 02139 (Dated: July 21 2017) We show how”
On learning to localize objects with minimal supervision (2014, unknown): “(see, e.g., Bach et al., 2012).”
An exact mapping between the Variational Renormalization Group and Deep Learning (2014, methodology): “By successively applying feature extraction, DNNs learn to deemphasize irrelevant features in the data while simultaneously learning relevant ones.”