Gradient-based algorithms with applications to signal-recovery problems

By Amir Beck, Marc Teboulle. 2010

428 citations recorded by the source corpus.

Subfields: Biomedical Engineering, Computational Mechanics, Mathematical Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Computational Mechanics

In Mathematical Physics

Citations in the corpus

Hyperparameter optimization with approximate gradient (2016, background): “As convergence criterion, we consider the norm of the (scaled) gradient mapping , a generalization of the gradient that ensures convergence to a stationary point (Beck and Teboulle, 2009, §1.4.4).”
Generalized Sparse Additive Models (2019, methodology): “In many cases, this algorithm has a per-iteration complexity of O ( np ) — precisely that of state-of-the-art algorithms for the lasso [Friedman et al., 2010, Beck and Teboulle, 2009b].”
A First Order Method for Solving Convex Bilevel Optimization Problems (2017, background): “Problem (P), which consists of minimizing the sum of a smooth function f and a possibly nonsmooth function g , is one of the most studied models in modern optimization with a huge body of literature (see, for instance, [4] and the references therein).”
Proximal Algorithms in Statistics and Machine Learning (2015, methodology): “If these conditions are met, then the proximal gradient method converges at rate 1/t with fixed step size γ = 1/γl (Beck and Teboulle, 2010).”
Smoothing and First Order Methods: A Unified Framework (2012, background): “Second, in many of these applications [9, 13], one of the nonsmooth terms in the model (PS), say, h2, plays a key role in describing a desirable property of the decision variable x which otherwise could be destroyed by smoothing.”
On the Convergence of Alternating Minimization for Convex Programming with Applications to Iteratively Reweighted Least Squares and Decomposition Schemes (2015, background): “An important and useful characterization of the prox operation is given in the following result (see, for example, [2]).”
Sparsity Constrained Nonlinear Optimization: Optimality Conditions and Algorithms (2012, methodology): “These conditions are then used to derive three numerical algorithms aimed at finding points satisfying the resulting optimality criteria: the iterative hard thresholding method and the greedy and partial sparse-simplex methods.”
Proximal Algorithms (2013, background): “Important papers on forward-backward splitting include those by Passty [159], Lions and Mercier [129], Fukushima and Mine [88], Gabay [90], Lemaire [120], Eckstein [78], Chen [54], Chen and Rockafellar [55], Tseng [184, 185, 187], Combettes and Wajs [62], and Beck and Teboulle [17, 18].”
On the Convergence of Block Coordinate Descent Type Methods (2013, methodology): “This was first shown by Nesterov in [11] and was later generalized for nonsmooth convex composite models in [12, 2, 3].”
"Convex Until Proven Guilty": Dimension-Free Acceleration of Gradient Descent on Non-Convex Functions (2017, methodology): “Adaptive restart accelerated gradient descent We use the accelerated gradient descent scheme of Beck and Teboulle (2009) with ωt = t/(t + 3).”
On stochastic proximal gradient algorithms (2014, methodology): “Proximal algorithms are well established optimization algorithms for dealing with non-smooth objective functions; Beck and Teboulle (2010); Parikh and Boyd (2013); Juditsky and Nemirovski (2012a,b).”
A Descent Lemma Beyond Lipschitz Gradient Continuity: First-Order Methods Revisited and Applications (2017, methodology): “α= 1), the number L is a Lipschitz continuity constant of the gradient of g, and we thus recover the classical global rate of the usual proximal gradient method [6].”
Majorization-Minimization Algorithms in Signal Processing, Communications, and Machine Learning (2017, methodology): “In this case, VMBF reduces to the proximal gradient algorithm (see [61] and [104] for examples).”