TIMELY: RTT-based Congestion Control for the Datacenter

By R. Mittal, V. Lam, Nandita Dukkipati, Emily R. Blem, Hassan M. G. Wassel, Monia Ghobadi, Amin Vahdat, Yaogong Wang, David Wetherall, David Zats. Computer communication review, 2015

838 citations recorded by the source corpus.

Datacenter transports aim to deliver low latency messaging together with high throughput. We show that simple packet delay, measured as round-trip times at hosts, is an effective congestion signal without the need for switch feedback. First, we show that advances in NIC hardware have made RTT measurement possible with microsecond accuracy, and that these RTTs are sufficient to estimate switch queueing. Then we describe how TIMELY can adjust transmission rates using RTT gradients to keep packet latency low while delivering high bandwidth. We implement our design in host software running over NICs with OS-bypass capabilities. We show using experiments with up to hundreds of machines on a Clos network topology that it provides excellent performance: turning on TIMELY for OS-bypass messaging over a fabric with PFC lowers 99 percentile tail latency by 9X while maintaining near line-rate throughput. Our system also outperforms DCTCP running in an optimized kernel, reducing tail latency by $13$X. To the best of our knowledge, TIMELY is the first delay-based congestion control protocol for use in the datacenter, and it achieves its results despite having an order of magnitude fewer RTT signals (due to NIC offload) than earlier delay-based schemes such as Vegas.

Subfields: Computer Networks and Communications, Information Systems

View Paper: https://www.semanticscholar.org/paper/327a02b19a60319cc35be860ad0259a5c1aef920

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

Citations in the corpus

PINT: Probabilistic In-band Network Telemetry (2020, neutral): “, RCP [22], XCP [40], TIMELY [53]; (3) Advanced routing, i.”
Copa: Practical Delay-Based Congestion Control for the Internet (2018, methodology): “We also implemented TIMELY [28], but it did not perform well in this setting (over 7 times worse than Copa on average), possibly because TIMELY is targeted at getting high throughput and low delay for long flows.”
Swift: Delay is Simple and Effective for Congestion Control in the Datacenter (2020, methodology): “Swift advances over TIMELY [38] include: decoupling fabric and host congestion; using a simple target end-to-end delay instead of RTT gradients; scaling the target based on load and topology; handling extreme incast; and measuring RTT precisely even in the presence of ACK coalescing.”
Datacenter RPCs can be General and Fast (2018, methodology): “example, DCQCN and Timely use up to 20- and 40-way incasts, respectively [68, 46].”
PCC Vivace: Online-Learning Congestion Control (2018, methodology): “, Timely [24] uses the RTT-gradient as a control signal similar to Vivace, but it still uses a hardwired rate control algorithm that maps events to fixed reactions (e.”
Congestion Control for Large-Scale RDMA Deployments (2015, background): “CNP generation: DCQCN is not particularly sensitive to congestion on the reverse path, as the send rate does not depend on accurate RTT estimation like TIMELY [31].”
Homa: a receiver-driven low-latency transport protocol using network priorities (2018, background): “t years. However, as our results will show, existing designs are sub-optimal for tail latency at high network load, particularly in networks with raw hardware latency in the single-digit microseconds [29,9,35,21]. In this section, we discuss the challenges that arise in such networks and we derive H”
Re-architecting datacenter networks and stacks for low latency and high performance (2017, background): “TIMELY [28], an alternative congestion control mechanism to DCQCN for lossless networks, relies exclusively on RTT as a congestion metric.”
RDMA over Commodity Ethernet at Scale (2016, background): “In order to reduce this collateral damage, ﬂow based congestion control mechanisms including QCN [13], DC-QCN [42] and TIMELY [27] have been introduced.”