Prox-PDA: The Proximal Primal-Dual Algorithm for Fast Distributed Nonconvex Optimization and Learning Over Networks

By Mingyi Hong, Davood Hajinezhad, Mingmin Zhao. 2017

169 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computational Mechanics, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Mechanics

In Computer Networks and Communications

Citations in the corpus

Robust and Communication-Efficient Collaborative Learning (2019, background): “Recently, there have been some works which study non-convex decentralized consensus optimization and establish convergence to a stationary point (Zeng and Yin, 2018; Hong et al., 2017, 2018; Sun and Hong, 2018; Scutari et al., 2017; Scutari and Sun, 2018; Jiang et al., 2017; Lian et al., 2017a).”
DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression (2019, methodology): “…adaptive distributed ADMM (Xu et al., 2017), adaptive distributed SGD (Cutkosky & Busa-Fekete, 2018), non-smooth distributed optimization (Sca-man et al., 2018), distributed proximal primal-dual algo-rithm (Hong et al., 2017), projection-free distributed online learning (Zhang et al., 2017b).”
On Nonconvex Decentralized Gradient Descent (2016, background): “Some typical applications include: (i) the distributed compressed sensing problems [14], [30], [39], [45], [49]; (ii) distributed consensus [9], [29], [55], [58], [61], [69]; (iii) distributed and parallel machine learning [15], [21], [33], [43], [55].”
Distributed Non-Convex First-Order optimization and Information Processing: Lower Complexity Bounds and Rate Optimal Algorithms (2018, methodology): “However, no lower bound analysis has been developed for distributed non-convex problem (19); there are even not many algorithms that provide achievable upper rate bounds (except for the recent works [4,20,41,42]), not to mention any analysis on the tightness/sharpness of these upper bounds.”
Gradient Primal-Dual Algorithm Converges to Second-Order Stationary Solutions for Nonconvex Distributed Optimization (2018, methodology): “Reference (Hong et al., 2017) proposes a primal-dual based method for unconstrained non-convex distributed optimization over a connected network (without a central controller), and derives the first global convergence rate for distributed non-convex optimization.”
Distributed nonconvex constrained optimization over time-varying digraphs (2018, methodology): “ no longer convex. Distributed nonconvex optimization: Distributed algorithms dealing with special instances of Problem (P) are scarce; they include primal methods [4,12,42,44] and dual-based schemes [21,53]. The key features of these algorithms are summarized in Table 1 and discussed next. Primal m”
A survey of distributed optimization (2019, methodology): “For the case where the agent dynamics are EL systems, continuous-time distributed algorithms have been developed in Meng et al. (2017) ; Qiu, Hong, and Xie (2016) and Zhang, Deng, and Hong (2017) .”