SCAFFOLD: Stochastic Controlled Averaging for Federated Learning

By Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, Ananda Theertha Suresh. 2019

4,191 citations recorded by the source corpus.

Subfields: Federated & Distributed Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Federated & Distributed Learning

Citations in the corpus

FedML: A Research Library and Benchmark for Federated Machine Learning (2020, methodology): “• Vertical Federated Learning (VFL).”
Personalized Federated Learning with Moreau Envelopes (2020, methodology): “While Assumption 1 is standard for convergence analysis, Assumptions 2 and 3 are widely used in FL context in which γ(2) f and σ 2 f quantify the sampling noise and the diversity of client’s data distribution, respectively [3, 8, 42, 43].”
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates (2020, background): “…with decentralized training: notably, high communication cost (Tang et al., 2018a; Wang et al., 2019; Koloskova et al., 2019), a need for time-varying topologies (Nedi´c & Olshevsky, 2014; Assran et al., 2019) and data-heterogeneity (Li et al., 2018; Karimireddy et al., 2019; Li et al., 2020a;b).”
Adaptive Federated Optimization (2020, methodology): “The control variates used by Karimireddy et al. (2019) allow them to derive convergence rates independent of the amount of heterogeneity.”
Personalized Federated Learning: A Meta-Learning Approach (2020, background): “…related to our paper, there are several works that study statistical heterogeneity of users’ data points in FL (Zhao et al., 2018; Sahu et al., 2018; Karimireddy et al., 2019; Haddadpour and Mahdavi, 2019; Khaled et al., 2019; Li et al., 2019), but they do not attempt to ﬁnd a personalized…”
Robust Aggregation for Federated Learning (2019, background): “Extensions were proposed in [7, 29, 33, 47, 50, 57, 69, 75, 79]; see also the recent surveys [46, 56].”
Adaptive Personalized Federated Learning (2020, methodology): “…updates the local version of the global model using its own local data, (iii) the server aggregates over the obtained local models to update the global model, and this process continues until convergence (McMahan et al., 2017; Mohri et al., 2019; Karimireddy et al., 2019; Pillutla et al., 2019).”
Federated Class-Incremental Learning (2022, background): “Federated learning (FL) [4, 18, 42, 46] providing secure privacy protection for local clients.”
Tighter Theory for Local SGD on Identical and Heterogeneous Data (2019, methodology): “…considered, for example with momentum (Yu et al., 2019a; Wang et al., 2019), with quantization (Reisizadeh et al., 2019; Basu et al., 2019), with adaptive stepsizes (Xie et al., 2019) and with various variance-reduction methods (Liang et al., 2019; Sharma et al., 2019; Karimireddy et al., 2019).”
Ditto: Fair and Robust Federated Learning Through Personalization (2020, methodology): “As a direct result of Theorem 10, we could state a result for Ditto when the global objective is FedAvg.”
From distributed machine learning to federated learning: a survey (2021, methodology): “The attention-augmented mechanism helps reduce the Table 2 Comparison among aggregation algorithms Algorithm Reg Fairness Heterogeneity Permutation C-E FedAvg N N N N N FedBCD S N N N S SAFL N S N N N FedMGDA+ N S N N S FedProx N N S N S FedMA N N N S S SCAFFOLD N N N N S FedAttOpt N N N N S Reg rep”
Federated Learning with Buffered Asynchronous Aggregation (2021, methodology): “Most previous works analyze synchronous federated learning methods, such as the works in (Lin et al., 2018; Li et al., 2018; Reddi et al., 2020; Li et al., 2020; Stich, 2019; Yu et al., 2019b; Li et al., 2019; Haddadpour and Mahdavi, 2019; Karimireddy et al., 2020).”