Optimizing data-intensive computations in existing libraries with split annotations
By Shoumik Palkar, Matei A. Zaharia. 2018
21 citations recorded by the source corpus.
Data movement between main memory and the CPU is a major bottleneck in parallel data-intensive applications. In response, researchers have proposed using compilers and intermediate representations (IRs) that apply optimizations such as loop fusion under existing high-level APIs such as NumPy and TensorFlow. Even though these techniques generally do not require changes to user applications, they require intrusive changes to the library itself: often, library developers must rewrite each function using a new IR. In this paper, we propose a new technique called split annotations (SAs) that enables key data movement optimizations over unmodified library functions. SAs only require developers to annotate functions and implement an API that specifies how to partition data in the library. The annotation and API describe how to enable cross-function data pipelining and parallelization, while respecting each function's correctness constraints. We implement a parallel runtime for SAs in a system called Mozart. We show that Mozart can accelerate workloads in libraries such as Intel MKL and Pandas by up to 15x, with no library modifications. Mozart also provides performance gains competitive with solutions that require rewriting libraries, and can sometimes outperform these systems by up to 2x by leveraging existing hand-optimized code.
Subfields: Computer Networks and Communications, Hardware and Architecture, Information Systems, Optimization & Training
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Networks and Communications
In Hardware and Architecture
- Yanping Huang
- Dmitry Lepikhin
- Noam Shazeer
- Maxim Krikun
- Shibo Wang
- Yuanzhong Xu
- Blake A. Hechtman
- HyoukJoong Lee
In Information Systems
In Optimization & Training
- Yonghui Wu
- Ruoming Pang
- Yanping Huang
- Yuanzhong Xu
- Maxim Krikun
- Dmitry Lepikhin
- Dehao Chen
- HyoukJoong Lee
Citations in the corpus
- GSPMD: General and Scalable Parallelization for ML Computation Graphs (2021, methodology): “proposed an annotation-based parallelization framework in [26] to speed up data-intensive compute libraries like Intel MKL and Pandas.”
- GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding (2020, neutral): “Software Software systems supporting neural networks evolved together with the advancement of the underlying hardware (Dean et al., 2012; Bastien et al., 2012; Abadi et al., 2016; Paszke et al., 2017; Palkar & Zaharia, 2019).”