Subgradient Methods

By Boyd. 2007

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

Citations in the corpus

Constrained Policy Optimization (2017, background): “In primal-dual optimization (PDO), dual variables are stateful and learned concurrently with the primal variables (Boyd et al., 2003).”
Dynamics of SGD with Stochastic Polyak Stepsizes: Truly Adaptive Variants and Convergence to Exact Solution (2022, methodology): “The classical Polyak stepsize [25] has been successfully used in the analysis of deterministic subgradient methods in different settings [5, 7, 18].”
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence (2020, methodology): “Contributions: Inspired by the classical Polyak step-size (Polyak, 1987) commonly used with the deterministic subgradient method (Hazan and Kakade, 2019; Boyd et al., 2003), we propose a novel adaptive learning rate for SGD.”
Deep Learning for Distributed Optimization: Applications to Wireless Resource Management (2019, methodology): “In the following sections, a constrained training algorithm is ﬁrst developed for solving (P1) with the long-term constraints based on the primal-dual method [6], [28].”
Energy Peer-to-Peer Trading in Virtual Microgrids in Smart Grids: A Game-Theoretic Approach (2020, unknown): “, ∇f (x)) if f (x) is differentiable [45]; in this case the subdifferential of D(λτ ) at λτ is ∂D(λτ ).”
Joint Optimization of Relay Strategies and Resource Allocations in Cooperative Cellular Networks (2006, unknown): “Subroutine 1 is guaranteed to converge to the optimal dual variable, if the step sizes ν are chosen following a diminishing step size rule [14].”
Energy-Efficient Resource Allocation in OFDMA Systems with Large Numbers of Base Station Antennas (2012, background): “Since the transformed problem for a given parameter q is concave in nature, it is guaranteed that the iteration between Layer 1 and Layer 2 converges to the optimal solution of (14) in the main loop, if the chosen step sizes satisfy the infinite travel condition [22], [24].”
Market-Based Coordination of Transmission and Generation Capacity Planning (2007, methodology): “Although the subgradient method may oscillate at final iterations, it is still an efficient method for solving the Lagrangian dual problem [29].”
Coordinated Charging of Electric Vehicles for Congestion Prevention in the Distribution Grid (2014, background): “In above algorithm [31], denotes the step size and can be chosen as which is a positive constant, independent of ; with such choice, the convergence is guaranteed.”
Wireless Information and Power Transfer: Energy Efficiency Optimization in OFDMA Systems (2013, methodology): “13In the literature, different methods such as exact line search and backtracking line search are commonly used for optimizing the step sizes [26], [28].”
Efficient multi-task feature learning with calibration (2014, unknown): “In the non-smooth case, the objective function value sequence { l ( W k ) generated by the sub-gradient method converges to the optimal value l ⋆ at a rate of O (1 / √ k ) under certain conditions [10], where l ( W ) denotes the objective function in Eq.”
Generalized Low Rank Models (2014, unknown): “(If any of these functions Lij are not differentiable, replace∇Lij by any subgradient of Lij [BXM03].”
Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants via the Mirror Stochastic Polyak Stepsize (2021, methodology): “In the deterministic setting Polyak’s results have been been successfully extended and used for solving weakly convex and smooth problems (Boyd et al., 2003; Davis et al., 2018; Hazan & Kakade, 2019).”
Decentralized Control of Connectivity for Multi-Agent Systems (2006, methodology): “The error can be limited in norm (see [19] for a similar proof):”
Constrained Deep Learning for Wireless Resource Management (2019, methodology): “In addition, ∇uy(u0) and ∂uy(u0) account for the gradient and the subgradient [9] of y(u) with respect to u evaluated at u = u0, respectively.”
A Joint Precoding Framework for Wideband Reconfigurable Intelligent Surface-Aided Cell-Free Network (2020, methodology): “We consider to solve (24) via the PDS method [37].”
Stochastic Gradient Descent with Polyak’s Learning Rate (2019, methodology): “A principled method for estimation, which is provably convergent with the same rate, is provided in [BM13, Chapter 4.2].”