This paper is included in the Proceedings of the 2022 USENIX Annual Technical Conference.

By Xuhao Luo, Weihai Shen, Shuai Mu, Tianyin Xu

Subfields: Software Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Yinmin Zhong
2. Lianmin Zheng
3. Ion Stoica
4. Woosuk Kwon
5. Shengyu Liu
6. Zhuohan Li
7. Joseph E. Gonzalez
8. Ying Sheng

Citations in the corpus

• Efficient Memory Management for Large Language Model Serving with PagedAttention (2023, methodology): “These systems utilize GPU kernel optimizations [1, 29, 31, 56], advanced batching mechanisms [14, 60], model parallelism [1, 41, 60], and parameter sharing [64] for efficient serving.”
• DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving (2024, methodology): “They range from general-purpose production-grade systems like TorchServe [7] and NVIDIA Triton [19] to systems optimized specifically for Transformer-based LLMs [9, 18, 21, 33, 50, 53, 54, 60].”