AuTO: scaling deep reinforcement learning for datacenter-scale automatic traffic optimization

By Li Chen, Justinas Lingys, Kai Chen, Feng Liu. Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, 2018

312 citations recorded by the source corpus.

Traffic optimizations (TO, e.g. flow scheduling, load balancing) in datacenters are difficult online decision-making problems. Previously, they are done with heuristics relying on operators' understanding of the workload and environment. Designing and implementing proper TO algorithms thus take at least weeks. Encouraged by recent successes in applying deep reinforcement learning (DRL) techniques to solve complex online control problems, we study if DRL can be used for automatic TO without human-intervention. However, our experiments show that the latency of current DRL systems cannot handle flow-level TO at the scale of current datacenters, because short flows (which constitute the majority of traffic) are usually gone before decisions can be made. Leveraging the long-tail distribution of datacenter traffic, we develop a two-level DRL system, AuTO, mimicking the Peripheral & Central Nervous Systems in animals, to solve the scalability problem. Peripheral Systems (PS) reside on end-hosts, collect flow information, and make TO decisions locally with minimal delay for short flows. PS's decisions are informed by a Central System (CS), where global traffic information is aggregated and processed. CS further makes individual TO decisions for long flows. With CS&PS, AuTO is an end-to-end automatic TO system that can collect network information, learn from past decisions, and perform actions to achieve operator-defined goals. We implement AuTO with popular machine learning frameworks and commodity servers, and deploy it on a 32-server testbed. Compared to existing approaches, AuTO reduces the TO turn-around time from weeks to ~100 milliseconds while achieving superior performance. For example, it demonstrates up to 48.14% reduction in average flow completion time (FCT) over existing solutions.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering, Information Systems

View Paper: https://www.semanticscholar.org/paper/69d1a6be560e728282ae3d93707a7ebfc55fd100

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

1. Chuanxiong Guo
2. Mosharaf Chowdhury
3. Kang G. Shin
4. Paul Patras
5. Hamed Haddadi
6. Myeongjae Jeon
7. Yibo Zhu
8. Juncheng Gu

In Electrical and Electronic Engineering

1. Kang G. Shin
2. Paul Patras
3. Chuanxiong Guo
4. Hamed Haddadi
5. Mosharaf Chowdhury
6. Chaoyun Zhang
7. Erik G. Larsson
8. Meysam Sadeghi

In Information Systems

1. Mosharaf Chowdhury
2. Kang G. Shin
3. Chuanxiong Guo
4. Hamed Haddadi
5. Yibo Zhu
6. Juncheng Gu

Citations in the corpus

• Deep Learning in Mobile and Wireless Networking: A Survey (2018, methodology): “shed light on automatic traffic optimization using a deep reinforcement learning approach [294].”
• Tiresias: A GPU Cluster Manager for Distributed Deep Learning (2019, methodology): “Instead of frequently solving an integer linear programming (ILP) [13] formulation or using a heavyweight deep learning mechanism [15], we leverage the classic foreground-background queueing idea [33], which has been shown to perform well for heavy-tailed distributions.”