Using Mpi Portable Parallel Programming With The Message Passing Interface

By C. Freytag. 2016

894 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Hardware and Architecture, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Hardware and Architecture

In Optimization & Training

Citations in the corpus

Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour (2017, methodology): “However, standard communication primitives like allreduce [11] perform summing, not averaging.”
Weight Agnostic Neural Networks (2019, methodology): “Our NumPy [111] implementation of NEAT [103] supports MPI [32] and OpenAI Gym [10] environments.”
Don't Use Large Mini-Batches, Use Local SGD (2018, methodology): “The MPI communication scheme (Gropp et al., 1999) is introduced for communication cost evaluation.”
FasterMoE: modeling and optimizing training of large-scale dynamic pre-trained models (2022, neutral): “The all-to-all operation in expert parallelism is commonly implemented by synchronous operators provided by communication libraries, such as MPI [6] or NCCL [8].”
kNN-IS: An Iterative Spark-based design of the k-Nearest Neighbors classifier for big data (2017, background): “Its application in data mining has been widely spread [12] [13] [14] , to the detriment of other parallelization schemes such as Message Passing Interface [15] , because of its fault-tolerant mechanism (recommendable for time-consuming tasks) and its ease of use [16] .”
A survey on platforms for big data analytics (2014, background): “One of the main features of MPI includes the state preserving process i.e., processes can live as long as the system runs and there is no need to read the same data again and again as in the case of other frameworks such as MapReduce (explained in section “ Apache hadoop ” ).”
Versatile, scalable, and accurate simulation of distributed applications and platforms (2014, neutral): “The simulation of parallel applications on parallel computing platforms has a long history in HPC, in particular in the context of applications based on MPI (Message Passing Interface) [41].”
Frame-Semantic Parsing (2014, methodology): “We did not use stochastic gradient descent for this data set as the number of training instances increased and parallelization of L-BFGS on a multicore setup implementing MPI (Gropp, Lusk, and Skjellum 1994) gave faster training speeds.”
A massively parallel tensor contraction framework for coupled-cluster computations (2014, methodology): “Parallelism is enabled to a limited extent by either using a multi-threaded BLAS library or by parallel MPI features of the program.”
Launchpad: A Programming Model for Distributed Machine Learning Research (2021, methodology): “As an alternative to the implicit communication mechanisms of modern numerical frameworks one can instead rely on low-level, distributed communication protocols such as message passing (e.g. MPI, Gropp et al., 1999) or a remote procedure call (RPC) mechanism (Birrell and Nelson, 1984).”