Convex Optimization Algorithms

By Dimitri P. Bertsekas. 2015

744 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Understanding Batch Normalization (2018, neutral): “Under our assumption E (cid:2) (cid:107)∇ (cid:96) i ( x ) − ∇ (cid:96) ( x ) 2 ≤ C we now have”
A Universal Catalyst for First-Order Optimization (2015, background): “The catalyst acceleration can be interpreted as a variant of the proximal point algorithm [3, 9], which is a central concept in convex optimization, underlying augmented Lagrangian approaches, and composite minimization schemes [5, 20].”
Coresets for Data-efficient Training of Machine Learning Models (2019, methodology): “However, in practice, the convergence rate of IG is known to be quite sensitive to the order of processing the functions (Bertsekas, 2015a; Gur-buzbalaban et al., 2017).”
A Stochastic Semismooth Newton Method for Nonsmooth Nonconvex Optimization (2018, background): “, [11, 35], which is beyond the scope of this paper.”
Catalyst Acceleration for First-order Convex Optimization: from Theory to Practice (2017, background): “t of the proximal point algorithm (Rockafellar, 1976; Guler, 1991), which is a central concept in convex optimization, underlying augmented Lagrangian approaches, and composite minimization schemes (Bertsekas, 2015; Parikh and Boyd, 2014). The proximal point algorithm consists of solving (1) by mini”
Multi-stage distributionally robust optimization with risk aversion (2021, methodology): “The basic idea of Algorithm 1 is outer linearization—cutting plane methods (see [6]).”
An introduction to continuous optimization for imaging (2016, methodology): “…for imaging problems is partly inspired by our own work and that of many colleagues, but also by important textbooks in optimization (Polyak 1987, Bertsekas 2015, Ben-Tal and 164 A. Chambolle and T. Pock Nemirovski 2001, Nesterov 2004, Boyd and Vandenberghe 2004, Nocedal and Wright 2006,…”
A Review of Safe Reinforcement Learning: Methods, Theories, and Applications (2024, unknown): “Based on Dankin’s Theorem and Convex Analysis [189], [17] provides theoretical support to the primal-dual (i.e., the Lagrange multiplier) method with a zero duality gap, which implies that the primal-dual method can be solved precisely in the dual domain.”
A Review of Safe Reinforcement Learning: Methods, Theory and Applications (2022, methodology): “Based on Dankin’s Theorem and Convex Analysis [29], [207] provides theoretical support to the primal-dual (i.e., the Lagrange multiplier) method with a zero duality gap, which implies that the primal-dual method can be solved exactly in the dual domain.”
Optimization Methods for Large-Scale Machine Learning (2016, methodology): “owed by a drastic slowdown after 1 or 2 epochs, is common in practice and is fairly well understood. An intuitive way to explain this behavior is by considering the following example due to Bertsekas [15]. Example 3.1. Suppose that each f i in (3.6) is a convex quadratic with minimal value at zero a”
Why random reshuffling beats stochastic gradient descent (2015, background): “One widely studied approach for solving problem (1) is the deterministic incremental gradient (IG) method [4–6].”
On the Convergence Rate of Incremental Aggregated Gradient Algorithms (2015, methodology): “As the performance of the determistic incremental methods are sensitive to the speciﬁc order chosen (see e.g. [2, Example 1.5.6], [4, Figure 2.1.9]), IAG can be slower than SAG if an unfavorable order is chosen and the analysis of IAG has to account for this worst-case scenarios.”