Programming Environments for Massively Parallel Distributed Systems

By K. M. Decker, R. Rehmann. 1994

Subfields: ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In ML Systems & Hardware

1. Sanjay Ghemawat
2. Michael Isard
3. Aakanksha Chowdhery
4. S. Hand
5. Joseph Dean
6. Hyeontaek Lim
7. Amir Salman Avestimehr
8. Suhas N. Diggavi

Citations in the corpus

• Pathways: Asynchronous Distributed Dataflow for ML (2022, methodology): “For example, most of today’s state-of-the-art ML workloads use a “single program multiple data” (SPMD) model, inspired by MPI (Clarke et al., 1994), where all accelerators run the same computation in lockstep and communication between accelerators is described by collectives like AllReduce.”
• Efficiently Scaling Transformer Inference (2022, methodology): “We use several communication collectives originating from MPI (Clarke et al., 1994).”