Harnessing smoothness to accelerate distributed optimization

By Guannan Qu, Na Li. IEEE Conference on Decision and Control, 2016

658 citations recorded by the source corpus.

Subfields: Computational Mechanics, Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/ef3c50a4d034d866e125d1d8340ba0842a124dc9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Networks and Communications

Citations in the corpus

On Nonconvex Decentralized Gradient Descent (2016, methodology): “When fi’s are convex, the existing algorithms include the (sub)gradient methods [8], [10], [24], [37], [40], [46], [59], [65], and the primal-dual domain methods such as the decentralized alternating direction method of multipliers (DADMM) [9], [51], [52], DLM [31], and EXTRA [53], [54].”
Multiagent Fully Decentralized Value Function Learning With Linear Convergence Rates (2018, background): “Several other works employ instead a gradient tracking strategy [27], [28], [33].”
Accelerating the Convergence Rates of Distributed Subgradient Methods with Adaptive Quantization (2018, methodology): “…attention during the last decade, mostly focusing on three classes of algorithms, namely, the alternating direction method of multipliers (ADMM) [1,17,33,35], distributed dual methods (mirror descent/dual averaging) [6,9,14,34,37], and distributed gradient algorithms [3–5,13,16,22,23,27,29,32,38].”
Balancing Communication and Computation in Distributed Optimization (2017, background): “In [29], [47], [59], [60], the authors also show their respective algorithms can achieve a linear rate of convergence.”
Push–Pull Gradient Methods for Distributed Optimization in Networks (2018, methodology): “, [5], [9]), the upperbound obtained here may be conservative.”
A Sharp Estimate on the Transient Time of Distributed Stochastic Gradient Descent (2019, background): “From Assumption 2, we have the following contraction property of W (see [20]).”
A Decentralized Proximal-Gradient Method With Network Independent Step-Sizes and Separated Convergence Rates (2017, background): “A number of recent works employed the so-called gradient tracking technique [51] in distributed optimization to conquer different issues [11,21,25,29,46].”
Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization (2017, background): “Furthermore, improved analysis of convergence rates over an undirected graph is available in [40] and [41].”
Achieving Geometric Convergence for Distributed Optimization Over Time-Varying Graphs (2016, methodology): “Finally, we note that reference [52], appearing simultaneously with this work, also proposed a method for distributed optimization based on gradient diﬀerences.”
A survey of distributed optimization (2019, methodology): “In the continuous-time setting, several distributed ﬁnite-time optimization algorithms have also been developed (see, e.g., Chen & Li, 2018; Feng & Hu, 2017; Lin et al., 2017; Pilloni, Pisano, Franceschelli, & Usai, 2016; Song & Chen, 2016 ).”
A Non-Asymptotic Analysis of Network Independence for Distributed Stochastic Gradient Descent (2019, background): “1) has been studied extensively in the literature under various distributed algorithms [43, 25, 26, 19, 15, 16, 39, 11, 35, 23, 45, 34], among which the distributed gradient descent (DGD) method proposed in [25] has drawn the greatest attention.”
Fast Convergence Rates of Distributed Subgradient Methods With Adaptive Quantization (2018, background): “The DSG algorithms for solving problem (1) have a long history, probably ﬁrst studied in [6] and recently received a wide attention (see, e.g., [1], [7]–[9], and the recent survey paper [4]).”
A review of cooperative multi-agent deep reinforcement learning (2019, methodology): “Then, in order to efficiently solve the problem, the authors combine the dynamic consensus [161] and the SAG algorithm [173].”
Accelerated Distributed Nesterov Gradient Descent (2017, methodology): “Instead, we use a gradient estimation sequence recently proposed in [24], [27], and [28], which is a more efficient use of consensus steps to get the correct descent direction.”
Distributed Gradient Methods for Convex Machine Learning Problems in Networks: Distributed Optimization (2020, methodology): “Simultaneously and independently of [29] and [30], the idea of tracking the gradients has been employed in [31], where a class of distributed algorithms have been developed for solving nonconvex problems in networks.”
Robust Distributed Accelerated Stochastic Gradient Methods for Multi-Agent Networks (2019, methodology): “ algorithm for the constrained reformulation of problem (1) (see Mokhtari and Ribeiro (2016) for this interpretation) that achieves exact convergence with linear rate to the optimal solution. Second, Qu and Li (2018) proposes to update the DG method such that agents also maintain, exchange, and comb”