MPI: A Message-Passing Interface Standard

By Message Passing Interface Forum. 1994

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In ML Systems & Hardware

Citations in the corpus

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding (2020, background): “Since the partitioner forces all the devices to run the same program, the communication patterns are also regular and XLA defines a set of collective operators that perform MPI-style communications [36].”
Automatic Cross-Replica Sharding of Weight Update in Data-Parallel Training (2020, background): “he corresponding operand. The result shape must be the same for all branch computations and this is the result shape of the Conditional operator. All-reduce. All-reduce has semantics to MPI Allreduce [18], which reduces a tensor across participating devices based on the provided binary reduction com”
Communication-Efficient Distributed Optimization of Self-Concordant Empirical Loss (2015, methodology): “Since the functions fi(w) can be accessed only locally, we consider distributed algorithms that alternate between a local computation procedure at each machine, and a communication round involving simple map-reduce type of operations [13, 32].”
A NICER View of PSR J0030+0451: Millisecond Pulsar Parameter Estimation (2019, methodology): “Facility: NICER (Gendreau et al. 2016) Software: Python/C language (Oliphant 2007), GNU Scientific Library (GSL; Gough 2009), NumPy (van der Walt et al. 2011), Cython (Behnel et al. 2011), SciPy (Jones et al. 2001–), OpenMP (Dagum & Menon 1998), MPI (Forum 1994), MPI for Python (Dalćın et al. 2008)”
A NICER View of PSR J0030+0451: Implications for the Dense Matter Equation of State (2019, unknown): “…2007), GNUScientific Library(GSL; Gough 2009), NumPy(van der Walt et al. 2011), Cython(Behnel et al. 2011), SciPy(Jones et al. 2001), MPI (Forum 1994), MPI for Python(Dalcín et al. 2008), Matplotlib (Hunter 2007; Droettboom et al. 2018), IPython(Perez & Granger 2007), Jupyter(Kluyver et…”
Stochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimization (2014, methodology): “Such a procedure can be achieved by a single round of communication, for example, using the Allreduce operation in MPI [20] or MapReduce [10].”
MPI for Python (2005, background): “MPI [5,6], the Message Passing Interface, is a standardized and portable message-passing system designed to function on a wide variety of parallel”
Fast lossless compression of scientific floating-point data (2006, methodology): “For example, we have included our algorithm in a message-passing interface (MPI) library [8].”