Proximal Splitting Methods in Signal Processing
By Patrick L. Combettes, Jean-Christophe Pesquet. 2009
2,764 citations recorded by the source corpus.
The proximity operator of a convex function is a natural extension of the notion of a projection operator onto a convex set. This tool, which plays a central role in the analysis and the numerical solution of convex optimization problems, has recently been introduced in the arena of inverse problems and, especially, in signal processing, where it has become increasingly important. In this paper, we review the basic properties of proximity operators which are relevant to signal processing and present optimization methods based on these operators. These proximal splitting methods are shown to capture and extend several well-known algorithms in a unifying framework. Applications of proximal methods in signal recovery and synthesis are discussed.
Subfields: Biomedical Engineering, Computational Mechanics, Mathematical Physics, Optimization & Training
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Biomedical Engineering
- Laurent Condat
- Stephen P. Boyd
- Thomas Pock
- Amir Beck
- Justin K. Romberg
- Antonin Chambolle
- Richard Baraniuk
- Neal Parikh
In Computational Mechanics
- Peter Richtárik
- Francis R. Bach
- Nicolas Loizou
- Simon Lacoste-Julien
- Jakub Konecný
- Mark W. Schmidt
- Julien Mairal
- Laurent Condat
In Mathematical Physics
- Laurent Condat
- Thomas Pock
- Stephen P. Boyd
- Francis R. Bach
- Antonin Chambolle
- Peter Richtárik
- Neal Parikh
- Stephen Becker
In Optimization & Training
- Peter Richtárik
- Simon Lacoste-Julien
- Jakub Konecný
- Nicolas Loizou
- Francis R. Bach
- Martin Jaggi
- Martin Takác
- Mark W. Schmidt
Citations in the corpus
- SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives (2014, methodology): “The same trick is commonly used to interpret Dijkstra’s set intersection as a primal algorithm instead of a dual block coordinate descent algorithm [12].”
- Natural Neural Networks (2015, background): “An equivalent proximal form of gradient descent [4] reveals the precise nature of ↵:”
- FreeLB: Enhanced Adversarial Training for Natural Language Understanding (2019, methodology): “Nonetheless, Madry et al. (2018) demonstrated that this saddle-point problem can be solved reliably with SGD for the outer minimization and PGD (a standard method for large-scale constrained optimization, see (Combettes & Pesquet, 2011) and (Goldstein et al., 2014)), for the inner maximization.”
- Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting (2015, background): “The classical deterministic proximal gradient approach [2], [36], [37] to solving (1) is to form a sequence via”
- Distributed optimization of deeply nested systems (2012, methodology): “The Z -step is new, however it always has the same form (4) of a “generalized” proximal operator (Rockafellar, 1976; Combettes and Pesquet, 2011).”
- Stochastic, Distributed and Federated Optimization for Machine Learning (2017, neutral): “The classical deterministic proximal gradient approach [11, 35, 136] to solving (4.”
- Learning with Submodular Functions: A Convex Optimization Perspective (2011, methodology): “adients (such as the logistic or square loss), while hwill only be assumed convex. Proximal methods have become increasingly popular over the past few years, both in the signal processing (see, e.g., [21, 208, 48] and numerousreferences therein) and in the machine learning communities (see, e.g., [1”
- A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization (2012, positive): “The proximal splitting algorithm (see, e.g., [9]) for nonsmooth optimization is also a special case of the BSUM algorithm.”
- Efficient Methods for Overlapping Group Lasso (2011, methodology): “Recently, a family of proximal splitting methods [27] has been proposed for converting a challenging optimization problem into a series of subproblems with a closed-form solution.”
- Multidimensional simultaneous random plus erratic noise attenuation and interpolation for seismic data by joint low-rank and sparse inversion (2015, background): “The inner updates within an ADMM iteration amount to evaluating so-called proximal operators (Moreau, 1965; Combettes and Pesquet, 2011; Parikh and Boyd, 2013), of which the one associated with the nuclear norm term, singular value thresholding, is the costliest because it involves a singular value ”
- FreeLB: Enhanced Adversarial Training for Language Understanding (2019, methodology): “Nonetheless, Madry et al. (2018) demonstrated that this saddle-point problem can be solved reliably with SGD for the outer minimization and PGD (a standard method for large-scale constrained optimization, see Combettes & Pesquet (2011) and Goldstein et al. (2014)), for the inner maximization.”
- Bayesian computation: a perspective on the current state, and sampling backwards and forwards (2015, methodology): “…book by Bauschke and Com-bettes (2011) for the mathematics underpinning proximal optimisation algorithms, and to the recent tutorial papers by Combettes and Pesquet (2011), Cevher et al. (2014) and Parikh and Boyd (2014) for an overview of the field and applications to signal processing and…”
- Multi-Agent Distributed Optimization via Inexact Consensus ADMM (2014, neutral): “For example, when , (20) has a closed-form solution known as the soft thresholding operator [29], [35]:”