Data center TCP (DCTCP)

By Mohammad Alizadeh, Albert G. Greenberg, David A. Maltz, J. Padhye, Parveen Patel, Balaji Prabhakar, Sudipta Sengupta, M. Sridharan. Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, 2010

2,361 citations recorded by the source corpus.

Cloud data centers host diverse applications, mixing workloads that require small predictable latency with others requiring large sustained throughput. In this environment, today's state-of-the-art TCP protocol falls short. We present measurements of a 6000 server production cluster and reveal impairments that lead to high application latencies, rooted in TCP's demands on the limited buffer space available in data center switches. For example, bandwidth hungry "background" flows build up queues at the switches, and thus impact the performance of latency sensitive "foreground" traffic. To address these problems, we propose DCTCP, a TCP-like protocol for data center networks. DCTCP leverages Explicit Congestion Notification (ECN) in the network to provide multi-bit feedback to the end hosts. We evaluate DCTCP at 1 and 10Gbps speeds using commodity, shallow buffered switches. We find DCTCP delivers the same or better throughput than TCP, while using 90% less buffer space. Unlike TCP, DCTCP also provides high burst tolerance and low latency for short flows. In handling workloads derived from operational measurements, we found DCTCP enables the applications to handle 10X the current background traffic, without impacting foreground traffic. Further, a 10X increase in foreground traffic does not cause any timeouts, thus largely eliminating incast problems.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering, Information Systems

View Paper: https://www.semanticscholar.org/paper/f7f38f4d4a0fc0e0da963b5166730e6321b6c171

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

In Information Systems

Citations in the corpus

Swift: Delay is Simple and Effective for Congestion Control in the Datacenter (2020, methodology): “We found protocols such as DCTCP [1] inadequate because they commonly experience milliseconds of tail latency, especially at scale.”
DeTail: reducing the flow completion time tail in datacenter networks (2012, methodology): “We compare the Microsoft’s measured distribution [13] with a synthetic normal one having a 50% larger median.”
AuTO: scaling deep reinforcement learning for datacenter-scale automatic traffic optimization (2018, methodology): “To make DRL scalable, we rst need to understand the long-tail distribution of datacenter tra c [3, 11, 33]: most of the ows are short ows 3 , but most of the bytes are from long ows.”
Presto: Edge-based Load Balancing for Fast Datacenter Networks (2015, methodology): “DCTCP [5] is a transport protocol that uses the portion of marked packets by ECN to adaptively adjust sender’s TCP’s congestion window to reduce switch buffer occupancy.”
Copa: Practical Delay-Based Congestion Control for the Internet (2018, methodology): “Many congestion-control algorithms for datacenters exploit the fact that one entity owns and controls the entire network, which makes it easier to incorporate in-network support [1, 3, 29, 36, 17].”
Sharing the Data Center Network (2011, methodology): “If switches support ECN, Seawall also incorporates the control laws from DCTCP [4] to further smooth out the sawtooth and reduce queue utilization at the bottleneck, resulting in reduced latency, less packet loss, and improved resistance against incast collapse.”
FlowRadar: A Better NetFlow for Data Centers (2016, methodology): “For example, in a production web search workload [13], 87.”
Understanding network failures in data centers: measurement, analysis, and implications (2011, background): “These properties have also been observed by others [4,5,12].”
Heracles: Improving resource efficiency at scale (2015, background): “There are also several networking protocols that prioritize short messages for LC tasks over large messages for BE tasks [5, 76].”
Re-architecting datacenter networks and stacks for low latency and high performance (2017, background): “Datacenters have evolved rapidly over the last few years, with Clos[1, 17] topologies becoming commonplace, and a new emphasis on low latency, first with improved transport protocols such as DCTCP[4] and more recently with solutions such as RDMA over Converged”
Congestion Control for Large-Scale RDMA Deployments (2015, methodology): “DCQCN is a rate-based, end-to-end congestion protocol, that builds upon QCN [17] and DCTCP [2].”
DRILL: Micro Load Balancing for Low-latency Data Center Networks (2017, background): “A common and vexing traffic pattern in datacenters is incast [20, 64] which is responsible for excessive congestion and packet loss [64].”
Homa: a receiver-driven low-latency transport protocol using network priorities (2018, methodology): “Recent years have seen numerous proposals for better transport protocols, including improvements to TCP [2, 3, 31] and a variety of new protocols [4, 6, 14, 15, 17, 25, 32].”
Trends in big data analytics (2014, methodology): “A recent variant of TCP, Data Center TCP (DCTCP) [6], addresses this artifact of the forkjoin structure of network traffic, by leveraging the ECN protocol extension implemented in most modern commodity switches, allowing end-to-end notification of network congestion without dropping packets.”
IX: A Protected Dataplane Operating System for High Throughput and Low Latency (2014, background): “We will also explore the synergies between IX and networking protocols designed to support microsecondlevel latencies and the reduced buffering characteristics of IX deployments, such as DCTCP [1] and ECN [52].”
Decentralized task-aware scheduling for data center networks (2014, methodology): “The ﬁrst two workloads are based on the Bing and Facebook traces discussed earlier ( § 2) while the third one models a more homogeneous application with ﬂow sizes that are uniformly distributed across [2KB, 50KB] (as suggested in prior work [4, 28, 17]).”