Quantized incremental algorithms for distributed optimization

By Michael G. Rabbat, Robert D. Nowak. 2005

407 citations recorded by the source corpus.

Subfields: Computational Mechanics, Computer Networks and Communications, Statistics and Probability, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Networks and Communications

In Statistics and Probability

In Optimization & Training

Citations in the corpus

Federated Learning: Strategies for Improving Communication Efficiency (2016, background): “Incremental, randomized and distributed optimization algorithms can be similarly analyzed in a quantized updates setting [16, 8, 7].”
Distributed Fixed Point Methods with Compressed Iterates (2019, methodology): “Among inexact methods, some provide algorithms that compute (sub)gradients at compressed iterates: [23, 24].”
Robust and Communication-Efficient Collaborative Learning (2019, methodology): “In the decentralized setting, quantization methods were proposed in different convex optimization contexts with nonvanishing errors (Yuksel and Basar, 2003; Rabbat and Nowak, 2005; Kashyap et al., 2006; El Chamie et al., 2016; Aysal et al., 2007; Nedic et al., 2008).”
On the Linear Convergence of the ADMM in Decentralized Consensus Optimization (2013, methodology): “Firstly, we establish the linear convergence rate of the ADMM that is applied to decentralized consensus optimization with strongly convex local objective functions.”
Stochastic, Distributed and Federated Optimization for Machine Learning (2017, background): “Incremental, randomized and distributed optimization algorithms can be similarly analyzed in a quantized updates setting [144, 66, 64].”
An Exact Quantized Decentralized Gradient Descent Algorithm (2018, methodology): “Note that the vanilla DGD method converges to a neighborhood of the optimal solution in the presence of quantization noise where the radius of convergence depends on the variance of quantization error [10], [11], [27], [30].”
Distributed learning in wireless sensor networks (2005, methodology): “First, [81] and [82] consider more general nonquadratic objective functions.”
Asynchronous Incremental Stochastic Dual Descent Algorithm for Network Resource Allocation (2017, background): “Strictly speaking, the message passing overhead is minimized if the updates occur along a Hamilton cycle [28].”
Quantized consensus ADMM for multi-agent distributed optimization (2015, methodology): “Methods that handle this constraint include quantized incremental algorithm [21], quantized dual averaging [22], and quantized subgradient method [16].”
Quantized Decentralized Consensus Optimization (2018, methodology): “Note that the vanilla DGD method converges to a neighborhood of the optimal solution in the presence of quantization noise where the radius of convergence depends on the variance of quantization error [9], [10], [26], [29].”
Nested Distributed Gradient Methods with Adaptive Quantized Communication (2019, methodology): “Our work is closely related to a few lines of research that we delineate below: 1) distributed first-order primal algorithms [9]–[11], [29]–[32]: methods that use only gradient information, 2) nested [11], [33]–[35]: methods that decompose the communication and computation steps, 3) communication ef”
Federated Learning With Cooperating Devices: A Consensus Approach for Massive IoT Networks (2019, background): “It belongs to the family of incremental gradient algorithms [26], [29] but, unlike these setups, optimization typically focuses on nonconvex objectives that are commonly”
Deep learning for situational understanding (2017, background): “they can be found with high probability in random geometric graphs that we use in our experiments [8], [9].”
Asynchronous gossip algorithms for stochastic optimization (2009, methodology): “Related to the algorithm of this paper is also the literature on incremental algorithms [4], [12], [14], [15], [17], [19], [20], [24], [26], [27], [30], where the network agents sequentially update a single iterate sequence and only one agent updates at any given time in a cyclic or a random order.”
Distributed Continuous-Time Convex Optimization With Time-Varying Cost Functions (2015, methodology): “The incremental subgradient algorithm is introduced as one of the earlier approaches addressing this problem [1], [2].”
Convergence Analysis of Quantized Primal-Dual Algorithms in Network Utility Maximization Problems (2016, background): “In [5], the authors proposed an incremental algorithm for solving a convex optimization problem.”