Asynchronous Byzantine Machine Learning ( the case of SGD ) Supplementary Material

Citations in the corpus

• How To Backdoor Federated Learning (2018, background): “A recent defense [14] requires the loss function to be Lipschitz and thus does not apply in general to large neural networks (See Sec.”
• Zeno++: Robust Fully Asynchronous SGD (2019, methodology): “We test the Byzantine-tolerance to the “sign-flipping” attack, which was proposed in (Damaskinos et al., 2018).”
• Fast and Secure Distributed Learning in High Dimension (2019, methodology): “Even-though the property of ( α, f ) –Byzantine resilience is a sufficient, but not a necessary condition for (weak) Byzantine resilience, it has been so far used as the defacto standard [1,4,11] to guarantee (weak) Byzantine resilience for SGD.”
• SGD: Decentralized Byzantine Resilience (2019, methodology): “In the latter, asynchrony is classically used in the context of SGD training , for algorithms where nodes (most often the workers ) are not always working on the same learning step [13, 17, 3].”
• Robust Federated Recommendation System (2020, background): “It appears that learning a robust FRS can directly employ existing defense strategies, i.e., utilize the model parameters among clients to detect Byzantine clients [7; 11; 23; 41; 21].”
• AGGREGATHOR: Byzantine Machine Learning via Robust Gradient Aggregation (2019, methodology): “…is the case in modern machine learning (Haykin, 2009; Mallat, 2016)), weak Byzantine resilience may lead to models with poor performance in terms of prediction accuracy, as a Byzantine worker can fool a provably converging SGD rule by leveraging a dimensional lee-way (El Mhamdi et al., 2018).”
• Fault Tolerance in Iterative-Convergent Machine Learning (2018, background): “A related body of work is distributed training under Byzantine faults (Blanchard et al., 2017; Chen et al., 2017; Damaskinos et al., 2018; Guerraoui et al., 2018), where a proportion of machines may act adversarially.”