Incremental subgradient methods for nondifferentiable optimization

By Angelia Nedić, Dimitri P. Bertsekas. 1999

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Federated learning of predictive models from federated Electronic Health Records (2018, methodology): “[18] However, IncrSub needs to deploy vanishing step size to reach exact convergence and only works over networks with a ring structure.”
Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling (2010, positive): “Subgradient descent on the average function f = 1 n =1 f i has identical convergence rate, as does the randomized version of incremental subgradient descent [NB01].”
Achieving Geometric Convergence for Distributed Optimization Over Time-Varying Graphs (2016, methodology): “Some earlier methods include distributed incremental (sub)gradient methods [37–39, 54 ] and incremental incremental proximal methods [ 3 , 68] , while a more recent Some distributed methods exist that do not (explicitly) use dual variables but can still converge to an exact consensual solution…”
Dataset Shift in Machine Learning (2009, methodology): “, L-BFGS), there is a large class of subgradient methods which can be applied in this case [e.g., see Shalev-Shwartz et al., 2007; Nedic and Bertsekas, 2001]”
Quantized incremental algorithms for distributed optimization (2005, background): “Proof of Theorem 3: This proof is similar to Nedi´ c and Bertsekas’s proofs of Proposition 2.1 in [ 4 ] and”
Asynchronous Incremental Stochastic Dual Descent Algorithm for Network Resource Allocation (2017, background): “The incremental subgradient descent and its variants have been widely applied to large-scale problems, and generally exhibit faster convergence than the traditional steepest descent algorithm and its variants [29].”
Tighter Bounds for Structured Estimation (2008, methodology): “For each inner optimization step, we used a fast-converging subgradientbased optimization algorithm with an adaptive Polyak-like step size [23].”
Distributed Sparse Linear Regression (2010, methodology): “Similarly, the schemes in [25] and [27] are based in incremental (sub)gradient methods [4], [21].”
Incremental Adaptive Strategies Over Distributed Networks (2007, methodology): “In order to resolve this difﬁculty, we call upon the concept of incremental gradient algorithms [11], [20], [21].”
Diffusion LMS Strategies for Distributed Estimation (2010, background): “Incremental solutions are useful for minimizing sums of convex functions as in (15) (see [5], [6], [12], [18]); they are based on the principle of iterating sequentially over each term, in some pre-defined order.”
Subgradient and sampling algorithms for l1 regression (2005, methodology): “The logn term in the runtime might be removable by using a sampling approach to estimate the subgradient, somewhat like the work of Nedi´c and Bertsekas.[ NB01 ]”
SMART: The Stochastic Monotone Aggregated Root-Finding Algorithm (2016, background): “These two structural assumptions, and the host of algorithms that adopt them, have led to big improvements in large-scale algorithm design (for examples, see [28,17,34,36,35,16,20,15,25,29,9,5,8,37,4, 27,6,31]).”