A geometrically convergent method for distributed optimization over time-varying graphs

23 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

A Unified Theory of Decentralized SGD with Changing Topology and Local Updates (2020, background): “This phenomena has been discussed (and sometimes been adressed by modifing the SGD updates) for example in (Shi et al., 2015; Lee et al., 2015; Nedić et al., 2016) and been rediscovered frequently in the context of stochastic optimization (Zhao et al.”
SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning (2019, methodology): “In another approach, option II in (6) which is known as gradient-tracking in decentralized algorithms (Shi et al., 2015; Nedich et al., 2016) uses By using an average of many stochastic gradients, option II has lower variance at the cost of slightly higher bias.”
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning (2019, methodology): “Further, they are used to obtain linearly converging decentralized algorithms under the guise of ‘gradient-tracking’ in (Shi et al., 2015; Nedich et al., 2016) and for gradient compression as ‘compressed-differences’ in (Mishchenko et al.”
Secure and Privacy-Preserving Consensus (2017, methodology): “To analyze the convergence speed of our approach, following the convergence definition in [30] and [31], we define the convergence rate as at least γ ∈ (0, 1) if there exists a positive constant C such that ∥∥x[k]−α1∥∥ ≤ Cγ holds for all k, where α is the average value of the initial states of all n”