Zeno: Distributed Stochastic Gradient Descent with Suspicion-based Fault-tolerance

By Cong Xie, Oluwasanmi Koyejo, Indranil Gupta. 2018

324 citations recorded by the source corpus.

Subfields: Optimization & Training, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Optimization & Training

In ML Systems & Hardware

Citations in the corpus

IBM Federated Learning: an Enterprise Framework White Paper V0.1 (2020, background): “Algorithms to improve robustness such as Krum [7], a Byzantine tolerant gradient descent coordinate-wise median fusion [14], and Zeno, a distributed stochastic gradient descent with suspicion-based fault-tolerance [32] are also available.”
Attack of the Tails: Yes, You Really Can Backdoor Federated Learning (2020, background): “Using robust estimation to defend against byzantine attacks has been studied extensively [66, 67, 68, 69, 70, 24, 71].”
Secure Byzantine-Robust Machine Learning (2020, background): “Further, [37] extends the Byzantine setting to attackers manipulating data transfer between workers and server and [38] extends it to tolerate an arbitrary number of Byzantine workers.”
RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasets (2018, unknown): “Leveraging additional data, (Xie, Koyejo, and Gupta 2018c) studies the trustworthy score-based schemes that guarantee ef-ﬁcient learning even when there is only one non-Byzantine worker, but additional data may not always be available in practice.”
Byzantine-Robust Learning on Heterogeneous Datasets via Resampling (2020, background): “• Gaussian : A Byzantine worker sends a Gaussian random vector of 0 mean and isotropic covariance matrix with standard deviation 200 (Xie et al., 2018a).”
FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping (2020, unknown): “, [9], [12], [29], [34], [44], [47], [48], [31]) that aim to tolerate Byzantine client failures.”
Byzantine-Resilient High-Dimensional SGD with Local Iterations on Heterogeneous Data (2020, unknown): “On the other hand, [CSX17,BMGS17,YCRB18,AAL18,SX19,XKG19b,YCRB19] provide robustness, but with no local iterations or sampling of clients; furthermore, they assume homogeneous (either same or i.i.d.) data across all clients.”
The Limitations of Federated Learning in Sybil Settings (2020, background): “Existing work has explored several ways to defend the FL training process [9, 50, 60, 61].”
Biscotti: A Blockchain System for Private and Secure Federated Learning (2021, background): “Prior work has demonstrated that adversaries can attack the shared model through poisoning attacks [11], [12], [15], [16], [17], [18], [19], in which an adversary contributes adversarial updates to shared model parameters.”
SLSGD: Secure and Efficient Distributed On-device Machine Learning (2019, background): “There are other robust SGD algorithms whose defense techniques are not based on robust stochastic [29].”
Byzantine-Robust Federated Machine Learning through Adaptive Model Averaging (2019, methodology): “To mitigate this limitation, different robust federated learning strategies have already been proposed in the literature (Blanchard et al. 2017; Mhamdi, Guerraoui, and Rouault 2018; Yin et al. 2018; Damaskinos et al. 2019; Chen, Su, and Xu 2017; Chen et al. 2018; Xie, Koyejo, and Gupta 2019b).”