Book Review : Parallel and Distributed Computation: Numerical Methods

By A. Nagumey. 1989

Subfields: Numerical Methods & Scientific Computing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Numerical Methods & Scientific Computing

Citations in the corpus

GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning (2019, background): “To this end, one can formulate the aforementioned problem as an average consensus problem that minimizes 1 N =1 f n ( θ n ) under the constraint θ n = θ (Bertsekas and Tisitsiklis, 1989).”
Data Encoding for Byzantine-Resilient Distributed Optimization (2019, methodology): “…they all par-allelly compute gradients on their local data and master aggregates them to update the parameter vector using gradient descent (GD) [BT89, Bot10, DCM + 12]; and (ii) model-parallelism architecture, where data points are partitioned across the features, and several worker nodes work…”
Privacy Aware Learning (2012, background): “Before doing so, we require some formalization of our notions of privacy and error measures, which we now provide.”
Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling (2010, neutral): “1) Partially Asynchronous Gossip Protocols: In a partially asynchronous iterative method, agents synchronize their iterations [1], which is the model of standard gossip, where computation proceeds in rounds, and in each round communication”
Distributed Stochastic Subgradient Projection Algorithms for Convex Optimization (2008, neutral): “Also related are the optimization algorithms in [2, 31].”
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms (1994, methodology): “Bertsekas & Tsitsiklis (1989) show that as long as each state is updated innnitely often and each action is tried an innnite number of times in each state, then the asynchronous algorithm eventually converges to the optimal value function.”
Stochastic, Distributed and Federated Optimization for Machine Learning (2017, background): “Optimization in distributed setting has been studied for decades, tracing back to at least works of Bertsekas and Tsitsiklis [14, 13, 177].”
Quantized Consensus (2006, background): “The problem in Section 1.2, without the integer constraints, has been studied in many forms, starting with Tsitsiklis (1984) and Bertsekas and Tsitsiklis (1997) and more recently by Blondel et al. (2005), Xiao and Boyd (2004), Xiao et al. (2005), Boyd et al. (2005), Olfati-Saber and Murray (2004),…”
A Reinforcement Learning Method for Maximizing Undiscounted Rewards (1993, background): “But it is well known that for successive approximation techniques the geometric rate of decay in the approximation error is proportional to , so that high values of can slow the convergence dramatically [2].”
Asynchronous Gradient Push (2018, background): “We assume that there is a set of times T = {t[1], t[2], t[3], .”
Motion Coordination with Distributed Information (2007, unknown): “The literature on automata theory and parallel computing [41, 42] presents precise notions of distributed algorithms for networks with ﬁxed topology.”
Convex Optimization in Signal Processing and Communications (2009, background): “The problem of reaching a consensus on a particular scalar value, or computing exact averages of the initial values of the agents, has attracted much recent attention as natural models of cooperative behavior in networked-systems (see Vicsek et al. [54], Jadbabaie et al. [20], Boyd et al. [8], Olfat”
Performance of a Distributed Stochastic Approximation Algorithm (2012, unknown): “To that goal, we write θ n as the sum of a vector in the consensus space and a disagreement vector.”
PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs (2012, unknown): “The merits of asynchronous computation have been studied extensively in the context of numerical algorithms [4].”
Distributed Control of Robotic Networks: A Mathematical Approach to Motion Coordination Algorithms (2009, background): “Together with this literature, the starting points for developing the material in this chapter are the standard notions of synchronous and asynchronous networks in distributed (Lynch, 1997; Peleg, 2000; Tel, 2001) and parallel (Bertsekas and Tsitsiklis, 1997; Parhami, 1999) computation.”
Alternating direction augmented Lagrangian methods for semidefinite programming (2010, methodology): “…direction methods for optimization problems arising from PDEs in [12], variational inequalities in [15] and a class of nonlinear optimization problems in [1] can be applied to our algorithm, we present here a diﬀerent and simple argument by formulating our algorithm as a ﬁxed point method.”
An online primal-dual method for discounted Markov decision processes (2016, methodology): “[3], by Borkar [7], by Bertsekas and Tsitsiklis [6]).”