Decentralized task-aware scheduling for data center networks

By Fahad R. Dogar, Thomas Karagiannis, Hitesh Ballani, Antony Ian Taylor Rowstron. Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, 2014

303 citations recorded by the source corpus.

Many data center applications perform rich and complex tasks (e.g., executing a search query or generating a user's news-feed). From a network perspective, these tasks typically comprise multiple flows, which traverse different parts of the network at potentially different times. Most network resource allocation schemes, however, treat all these flows in isolation -- rather than as part of a task -- and therefore only optimize flow-level metrics. In this paper, we show that task-aware network scheduling, which groups flows of a task and schedules them together, can reduce both the average as well as tail completion time for typical data center applications. To achieve these benefits in practice, we design and implement Baraat, a decentralized task-aware scheduling system. Baraat schedules tasks in a FIFO order but avoids head-of-line blocking by dynamically changing the level of multiplexing in the network. Through experiments with Memcached on a small testbed and large-scale simulations, we show that Baraat outperforms state-of-the-art decentralized schemes (e.g., pFabric) as well as centralized schedulers (e.g., Orchestra) for a wide range of workloads (e.g., search, analytics, etc).

Subfields: Computer Networks and Communications, Information Systems

View Paper: https://www.semanticscholar.org/paper/f24fb2ab21266ba437ad650ac405cb56650fef3d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

Citations in the corpus

Programmable Packet Scheduling at Line Rate (2016, background): “Past work has demonstrated significant performance benefits resulting from switch support for flexible allocation [12, 21, 32, 39].”
Efficient coflow scheduling with Varys (2014, unknown): “Simple scheduling mechanisms like FIFO and its variants [15,19], which are attractive for the ease of decentralization, do not perform well in such an environment – one large coﬂow can slow down many smaller ones or result in many missed deadlines.”
Efficient Coflow Scheduling Without Prior Knowledge (2015, methodology): “Hence, unlike existing work [19, 20, 25], we do not assume anything about a coflow’s characteristics like the number of flows, endpoints, or waves, the size of each flow, their arrival times, or the presence of barriers.”
Hopper: Decentralized Speculation-aware Cluster Scheduling at Scale (2015, background): “, see [12, 14, 25].”
A link layer protocol for quantum networks (2019, unknown): “[24, 37]).”
Low Latency Geo-distributed Data Analytics (2015, background): “Orchestra [25], Varys [26], and Baraat [30] are network flow schedulers that optimize for completion time of coflows, i.”