Fault-Tolerant Multi-Agent Optimization: Optimal Iterative Distributed Algorithms

By Lili Su, Nitin H. Vaidya. 2016

152 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Optimization & Training, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Optimization & Training

In ML Systems & Hardware

Citations in the corpus

Advances and Open Problems in Federated Learning (2019, background): “ such as median-based aggregators [101, 426], Krum [69], and trimmed mean [426]. Past work has shown that various robust aggregators are provably effective for Byzantine-tolerant distributed learning [372, 69, 101] under appropriate assumptions, even in federated settings [326, 415]. Despite this, F”
Zeno++: Robust Fully Asynchronous SGD (2019, background): “In particular, tolerance to Byzantine failures (Blanchard et al., 2017; Chen et al., 2017; Yin et al., 2018; Feng et al., 2014; Su & Vaidya, 2016; 2018; Xie et al., 2019b; Alistarh et al., 2018; Cao & Lai, 2018; Xie et al., 2019c) has become an important topic in the distributed machine learning…”
Byzantine Stochastic Gradient Descent (2018, background): “Su and Vaidya [26, 27] consider a similar setting: in Su and Vaidya [26] they focus on the single-dimensional (d = 1) case, whereas Su and Vaidya [27] considers the multidimensional setting, but only consider a restricted family of consensus-based algorithms.”
Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates (2018, methodology): “Other methods have been proposed for Byzantine-robust distributed learning and optimization; e.g., Su and Vaidya (2016a,b).”
Zeno: Byzantine-suspicious stochastic gradient descent (2018, methodology): “For example, Chen et al. [4], Su and Vaidya [17, 18] use geometric median as the aggregation rule.”
Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulation (2019, background): “Most of the existing Byzantine-tolerant machine-learning algorithms (Blanchard et al., 2017; Chen et al., 2017; Yin et al., 2018; Feng et al., 2014; Su & Vaidya, 2016a;b; Alistarh et al., 2018) focus on the protection of distributed Stochastic Gradient Descent (SGD).”
Byzantine-Tolerant Machine Learning (2017, background): “A closer approach to ours has been recently proposed in [20,21].”
The Convergence of Stochastic Gradient Descent in Asynchronous Shared Memory (2018, background): “In a series of papers [30, 32], Su and Vaidya have considered the problem of adding fault-tolerance to the problem of multi-agent optimization, as well as non-Bayesian optimization under asynchrony and crash failures [31].”
Zeno++: robust asynchronous SGD with arbitrary number of Byzantine workers (2019, background): “For this reason, fault tolerance in distributed settings has attracted increasing attention in the machine-learning community [1, 2, 3, 5, 13, 14, 15, 16].”
Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learning (2018, background): “This Byzantine-robust distributed learning problem has attracted attention in a recent line of work [3, 10, 17, 26, 61, 62, 69].”
On the Robustness of a Neural Network (2017, methodology): “This coarse-grained approach is the one taken by recent works on the robustness of machine learning [20], [21], [22].”
Defending Distributed Systems Against Adversarial Attacks: Consensus, Consensusbased Learning, and Statistical Learning (2020, background): “(1) Multi-Agent Optimization: In [15, 17] and a series of technical reports, we considered the problem of multi-agent optimization wherein an unknown subset of agents suffer Byzantine faults.”
Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent (2017, background): “In [28], the study only deals with parameter vectors of dimension one, which is too restrictive for today’s multi-dimensional machine learning.”
Securing Distributed Machine Learning in High Dimensions (2018, background): “Unfortunately, even a single Byzantine worker can completely skew the empirical risk function and thus foils the whole empirical risk minimization approach [SV16].”
Zeno: Distributed Stochastic Gradient Descent with Suspicion-based Fault-tolerance (2018, methodology): “Despite their differences, the existing majority-based methods for synchronous SGD (Blanchard et al., 2017; Chen et al., 2017; Yin et al., 2018; Su & Vaidya, 2016a; Alistarh et al., 2018) assume that the non-faulty workers dominate the entire set of workers.”