Convex optimization with sparsity-inducing norms

By Francis R. Bach, Rodolphe Jenatton, Julien Mairal, Guillaume Obozinski. 2011

347 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Numerical Analysis

View Paper: https://www.semanticscholar.org/paper/8280079306891aefeb9a12368b7190bc62a5a210

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Numerical Analysis

Citations in the corpus

Decimeter-Level Localization with a Single WiFi Access Point (2016, background): “Indeed, it has been well-studied in optimization theory that minimizing the L-1 norm of a vector favors sparse solutions for that vector [7].”
Fusing Multi-Stream Deep Networks for Video Classification (2015, background): “Since it can be analytically solved in linear time [3], the above optimization process is fairly efficient.”
Fast Stochastic Methods for Nonsmooth Nonconvex Optimization (2016, background): “This paper builds upon our recent series of papers on fast stochastic methods for smooth nonconvex optimization [22, 23], with a novel analysis for nonconvex and nonsmooth functions.”
Optimization with Sparsity-Inducing Penalties (2011, background): “Some of the material from this monograph is taken from an earlier book chapter [12] and the dissertations of Rodolphe Jenatton [65] and Julien Mairal [85].”
Optimizing the Latent Space of Generative Networks (2017, background): “For instance, in the sparse coding literature, it is common practice to set the representation space to the closed unit ball B (1 , d, p ) [Bach et al., 2011], where Since we are interested in matching the the properties of GANs, we make similar choices to them when it comes to the representation…”
Studying Very Low Resolution Recognition Using Deep Networks (2016, methodology): “For each class, 1,000 synthetic text images were rendered for training, with a normalized height of 105.”
Practice and Theory of Blendshape Facial Models (2014, background): “In other words, each vertex in the basis is encouraged to be zero, but if the vertex is not zero then there is no further penalization of its components [BJMO12].”
Non-intrusive human activity monitoring in a smart home environment (2013, background): “A wide range of efficient algorithms has been proposed to solve this problem [15].”
A Regularized Linear Dynamical System Framework for Multivariate Time Series Analysis (2015, methodology): “(5), such as (Yuan, Liu, and Ye 2011; Qin, Scheinberg, and Goldfarb 2013), however, the second order optimization methods, like SOCP, always get solutions with high precision (low duality gap) (Bach et al. 2011).”
Linear dimensionality reduction: survey, insights, and generalizations (2014, background): “While the `2 case can be solved in closed form as an augmented least squares, the `1 case requires a quadratic program (Tibshirani, 1996); though the simple quadratic program formulation scales poorly (Boyd et al., 2011; Bach et al., 2011).”
Efficient Multi-label Classification with Many Labels (2013, background): “However, even with the recent advances in learning with structured sparsity (Bach et al., 2011), the involved optimization problem is still computationally expensive, especially when there are a lot of labels to select from.”
Convex Multitask Learning with Flexible Task Clusters (2012, methodology): “In recent years, accelerated proximal methods (Nesterov, 2007) have been popularly used by the machine learning community (Bach et al., 2011) for convex problems of the form min θ f ( θ )+ r ( θ ) , where f ( θ ) is convex and smooth, and r ( θ ) is convex but nonsmooth.”
Automatic Feature Learning for Spatio-Spectral Image Classification With Sparse SVM (2014, methodology): “and last objective by proposing the use of a sparse 1 linear support vector machine (SVM) [40], which naturally performs”
Convergence Rates of Inexact Proximal-Gradient Methods for Convex Optimization (2011, background): “However, in many scenarios the proximity operator may not have an analytic solution, or it may be very expensive to compute this solution exactly.”
Probabilistic Multi-Label Classification with Sparse Feature Learning (2013, positive): “The closed-form solutions of the convex optimization problems with sparsityinducing norms in (11) and (12) can be computed by applying group soft-thresholding and direct soft-thresholding operators [Bach et al., 2011] respectively, such as”
Making the Most of Bag of Words: Sentence Regularization with Alternating Direction Method of Multipliers (2014, methodology): “For this update, we apply a proximal gradient method (Bach et al., 2011), since is convex and continuously differentiable, and Ω las ( w ) is a convex function whose proximal operator (Moreau, 1963) can be evaluated efﬁciently.”
Parallel Direction Method of Multipliers (2014, background): “Due to the increasing interest in structural sparsity [1], composite regularizers have become widely used, e.”