Understanding data center traffic characteristics

By Theophilus A. Benson, Ashok Anand, Aditya Akella, Ming Zhang. Workshop on Research on Enterprise Networking, 2009

718 citations recorded by the source corpus.

As data centers become more and more central in Internet communications, both research and operations communities have begun to explore how to better design and manage them. In this paper, we present a preliminary empirical study of end-to-end traffic patterns in data center networks that can inform and help evaluate research and operational approaches. We analyze SNMP logs collected at 19 data centers to examine temporal and spatial variations in link loads and losses. We find that while links in the core are heavily utilized the ones closer to the edge observe a greater degree of loss. We then study packet traces collected at a small number of switches in one data center and find evidence of ON-OFF traffic behavior. Finally, we develop a framework that derives ON-OFF traffic parameters for data center traffic sources that best explain the SNMP data collected for the data center. We show that the framework can be used to evaluate data center traffic engineering approaches. We are also applying the framework to design network-level traffic generators for data centers.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/022a0317d5bf2b38847b03f7c9bc3bfa35950199

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

Citations in the corpus

A Survey on Optical Interconnects for Data Centers (2012, background): “There are several research papers that have investigated the data center traffic such as the ones presented by Microsoft Research ([23],[24],[25]).”
MicroTE: fine grained traffic engineering for data centers (2011, methodology): “[6, 5], show that the arrival processes for data center traffic follows an ON-OFF pattern whos e”
On the impact of packet spraying in data center networks (2013, background): “Further, it has high signaling and connection establishment complexity for short flows, which typically dominate the data center environment [2], [3].”
The Impact of Virtualization on Network Performance of Amazon EC2 Data Center (2010, background): “A more recent study [4] has measured the network traffic workload of production data centers.”
Data Center Network Virtualization: A Survey (2013, background): “It is known that data center network trafﬁc exhibits different characteristics than the trafﬁc in traditional data networks [74].”
Joint VM placement and routing for data center traffic engineering (2012, methodology): “We construct a synthesized workload by emulating the DC traffic based on existing traffic measurement studies [21], [22], [23], [24], [25].”
c-Through: part-time optics in data centers (2010, background): “The bandwidth at the top of the tree is typically a fraction of the incoming capacity, creating a bottleneck.”
Inside the Social Network's (Datacenter) Network (2015, background): “observe that packet arrivals exhibit an on/off pattern at the end-host level [12, 13].”
End-to-end Performance Isolation Through Virtual Datacenters (2014, background): “known [15, 39], most workloads exhibit bursty behavior at very fine timescales (below 1 sec); hence, using a very short control interval leads to large estimation errors.”
A Survey on Network Methodologies for Real-Time Analytics of Massive IoT Data and Open Research Issues (2017, background): “For instance, according to the work in [164], approximately 60% of the core and edge links are active at any given time in a DCN.”
The nature of data center traffic: measurements & analysis (2009, background): “Benson et al. [ 7 ] perform a complementary study of tra�c atthe edges of adatacenterbyexamining SNMPtracesfromrouters and identify ON-OFF characteristics whereas this paper examines novel aspects of tra�c within a data center in detail.”
Network traffic characteristics of data centers in the wild (2010, background): “Despite the growing applicability of data centers in a wide variety of scenarios, there are very few systematic measurement studies [19, 3] of data center usage to guide practical issues in data center operations.”
Hedera: Dynamic Flow Scheduling for Data Center Networks (2010, background): “These cluster applications can often be bottlenecked on the network, not by local resources [4, 7, 9, 14, 16].”
Consolidating virtual machines with dynamic bandwidth demand in data centers (2011, background): “At the chassis/rack level, the aggregate rate must not exceed the capacity of the end-of-row/top-of-rack switch.”
A roadmap for traffic engineering in SDN-OpenFlow networks (2014, background): “Such difference in traffic burstiness leads to high packet loss rates of the underutilized edge and aggregation links, and low packet loss rates of highly utilized core links [66,67].”
Less Is More: Trading a Little Bandwidth for Ultra-Low Latency in the Data Center (2012, background): “Most data center networks operate at loads less than 30% [9], so a load of 60% with Poisson/bursty traffic is highly unlikely—the performance degradation would be too severe.”
DRILL: Micro Load Balancing for Low-latency Data Center Networks (2017, background): “The bulk of microbursts responsible for most packet loss, for instance, last for a few microseconds in measurements of [24].”