Partial Solutions Manual Parallel and Distributed Computation : Numerical Methods

By Dimitri P. Bertsekas, John N. Tsitsiklis. 1997

Subfields: Numerical Methods & Scientific Computing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Numerical Methods & Scientific Computing

Citations in the corpus

Optimal Distributed Online Prediction Using Mini-Batches (2010, background): “In an asynchronous distributed computing environment (see, e.g., Tsitsiklis et al., 1986; Bertsekas and Tsitsiklis, 1989), this can be quite wasteful.”
CoCoA: A General Framework for Communication-Efficient Distributed Optimization (2016, methodology): “…for updating the parameter vector α in an iterative fashion include the Jacobi method, in which updates made to coordinates of α do not take into account the most recent updates to the other coordinates, and Gauss-Seidel, in which the most recent information is used (Bersekas and Tsitsiklis, 1989).”
Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization (2015, methodology): “It has been successfully applied to the distributed neural network [Dean et al., 2012] and the parameter server [Li et al., 2014a] to solve deep neural network.”
An asynchronous parallel stochastic coordinate descent algorithm (2013, methodology): “Unlike the asynchronous scheme in Bertsekas and Tsitsiklis [1989, Section 6.1], the latest value of x is updated at each step, not an earlier iterate.”
Coordinate descent algorithms (2015, methodology): “In the terminology of Bertsekas and Tsitsiklis [5], the algorithm is totally asynchronous if (a) each index i ∈ { 1 , 2 , . . . , n } of x is updated at inﬁnitely many iterations; and (b) if ν kj denotes the iteration at which component j of the vector ˆ x k was last updated, then ν kj → ∞ as k → ∞…”
On the Linear Convergence of the ADMM in Decentralized Consensus Optimization (2013, methodology): “Deﬁning x ∈ R LN as a vector concatenating all x i , z ∈ R 2 EN as a vector concatenating all z ij , and can be written in a matrix form as where g ( z ) = 0 , which ﬁts the form of (2), and is amenable to the ADMM.”
Optimal Algorithms for Smooth and Strongly Convex Distributed Optimization in Networks (2017, methodology): “Standard methods as described by [23] can be used for performing this parallelization of gradient computations.”
A reliable effective terascale linear learning system (2011, background): “Some prior works (McDonald et al., 2010; Zinkevich et al., 2010) consider online learning with averaging and Duchi et al. (2010a) propose gossip-style message passing algorithms extending the existing literature on distributed convex optimization (Bertsekas and Tsitsiklis, 1989).”
Distributed GraphLab: A Framework for Machine Learning in the Cloud (2012, background): “For example, linear systems (common to many MLDM algorithms) have been shown to converge faster when solved asynchronously [4].”
ASAGA: Asynchronous Parallel SAGA (2016, methodology): “The seminal textbook of Bertsekas and Tsitsiklis (1989) provides most of the foundational work for parallel and distributed optimization algorithms.”
Achieving Geometric Convergence for Distributed Optimization Over Time-Varying Graphs (2016, methodology): “The resulting algorithms include distributed dual decomposition [65] and decentralized alternating direction method of multipliers (ADMM) [5, 32].”
Stochastic Gradient-Push for Strongly Convex Functions on Time-Varying Directed Graphs (2014, background): “This is a generalization of a relation from [3], Section 7.”
Asynchronous Decentralized Accelerated Stochastic Gradient Descent (2018, background): “Following the seminal work [10], many research efforts have been devoted to study communication-efficient decentralized algorithms.”
Reinforcement Learning: An Introduction (1998, methodology): “Bertsekas and Tsitsiklis (1989) provide excellent coverage of these variations and their performance di↵erences.”
Diffusion Strategies Outperform Consensus Strategies for Distributed Estimation Over Adaptive Networks (2012, methodology): “In this article, we study three cooperative strategies for distributed estimation.”
Decentralized, Adaptive Coverage Control for Networked Robots (2009, background): “…yielded to rigorous mathematical treatment first in the distributed and parallel computing community (Tsitsiklis 1984 Tsitsiklis et al. 1986 Bertsekas and Tsitsiklis 1989, 2007) in discrete time, and more recently in the controls community in continuous time (Jadbabaie et al. 2003…”
Harnessing smoothness to accelerate distributed optimization (2016, background): “The early work of this problem can be found in [4], [5].”
Convergence Rate Analysis of Several Splitting Schemes (2014, background): “For special choices of A and B, operator-splitting schemes naturally give rise to algorithms with parallel or distributed implementations [6,8].”