First-Order Methods in Optimization

By Amir Beck. 2017

2,219 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/b532b74b9112e5c37a72ad6de9cc618c48b0467a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Numerical Analysis

Citations in the corpus

Distributed Proximal Splitting Algorithms with Rates and Acceleration (2020, background): “Proximal splitting algorithms [1, 2, 7, 10, 15, 20] are particularly well suited for large-scale convex nonsmooth optimization.”
Stability of Stochastic Gradient Descent on Nonsmooth Convex Losses (2020, background): “ by its excess population risk εrisk(A) := FD(A(S)) −FD(x∗). 2For equivalence to hold it is necessary that the function is well-deﬁned and satisﬁes (2) over an open set containing X, see Thm. 3.61 in Beck (2017). We will assume this is the case, which can be done w.l.o.g.. 5 Note that εrisk(A) is a ”
Walking in the Shadow: A New Perspective on Descent Directions for Constrained Minimization (2020, neutral): “, [35]), we know that ∇φ∗(y) = argmaxx∈P{⟨y, x⟩ − φ(x)}.”
Optimality and Approximation with Policy Gradient Methods in Markov Decision Processes (2019, methodology): “We present the proof in Appendix D, which utilizes standard non-convex optimization results (e.g. [Beck, 2017, Theorem 10.15]) for the convergence of projected gradient ascent on smooth functions over a convex set.”
Gradient Estimation with Stochastic Softmax Tricks (2020, neutral): “[10] Amir Beck.”
A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approach (2019, methodology): “The update of PP method for minimizing a convex function h is deﬁned as where η is a positive scalar (Bertsekas, 1999; Beck, 2017).”
Dualize, Split, Randomize: Fast Nonsmooth Optimization Algorithms (2020, methodology): “1 Related Work Splitting algorithms: Algorithms allowing to solve nonsmooth optimization problem involving several proximable terms are called proximal splitting algorithms [6, 7, 19, 21, 25, 44, 57].”
Unified Analysis of Stochastic Gradient Methods for Composite Convex and Smooth Optimization (2020, neutral): “This is classic result, see for example the “Second Prox Theorem” in Section 6.5 in [6].”
Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants via the Mirror Stochastic Polyak Stepsize (2021, methodology): “SMD is often analyzed as a stochastic method for optimizing non-smooth Lipschitz continuous convex functions (Nemirovski et al., 2009; Bubeck, 2015; Beck, 2017).”
Accelerated Linear Convergence of Stochastic Momentum Methods in Wasserstein Distances (2019, neutral): “[Bec17] A.”
A Conditional Gradient-based Method for Simple Bilevel Optimization with Convex Lower-level Problem (2022, background): “In fact, this is also common in the literature on constrained optimization, where the generated iterate could be infeasible and thus f(x̂)− f∗ could be negative (see, e.g., Beck (2017)).”
DINGO: Distributed Newton-Type Method for Gradient-Norm Optimization (2019, background): “First-order methods [10], e.g., stochastic gradient descent (SGD) [11], solely rely on gradient information and as a result are rather easy to implement in distributed settings.”
A regularization interpretation of the proximal point method for weakly convex functions (2020, background): “The importance of large scale optimization problems in machine learning has led to a resurgence of interest in first order optimization methods [8], and stochastic gradient descent in particular [9].”