AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving

By Zhuohan Li, Lianmin Zheng, Yinmin Zhong, Vincent Liu, Ying Sheng, Xin Jin, Yanping Huang, Z. Chen, Hao Zhang, Joseph E. Gonzalez and 1 more

Subfields: Efficiency & Compression, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Efficiency & Compression

1. Ion Stoica
2. Joseph E. Gonzalez
3. Xin Jin
4. Lianmin Zheng
5. Yinmin Zhong
6. Ying Sheng
7. Xuanzhe Liu
8. Zhuohan Li

In ML Systems & Hardware

1. Joseph E. Gonzalez
2. Ion Stoica
3. Xin Jin
4. Lianmin Zheng
5. Yibo Zhu

Citations in the corpus

• Efficient Memory Management for Large Language Model Serving with PagedAttention (2023, methodology): “Therefore, it is necessary to partition them across distributed GPUs and execute them in a model parallel fashion [28, 63].”
• Splitwise: Efficient Generative LLM Inference Using Phase Splitting (2023, methodology): “LLM inference serving is a rapidly developing field, with several recent works optimizing batching [23], [25], [51], [53], [81], scheduling [22], [42], [51], [66], [73], [79], and memory usage [32], [35], [51], [74].”
• DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving (2024, methodology): “Although short-term interval is impossible to predict, the workload pattern over longer timescales (e.g., hours or days) is often predictable [33, 55].”