A scalable, commodity data center network architecture

By Mohammad Al-Fares, Alexander Loukissas, Amin Vahdat. Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, 2008

3,857 citations recorded by the source corpus.

Today's data centers may contain tens of thousands of computers with significant aggregate bandwidth requirements. The network architecture typically consists of a tree of routing and switching elements with progressively more specialized and expensive equipment moving up the network hierarchy. Unfortunately, even when deploying the highest-end IP switches/routers, resulting topologies may only support 50% of the aggregate bandwidth available at the edge of the network, while still incurring tremendous cost. Non-uniform bandwidth among data center nodes complicates application design and limits overall system performance. In this paper, we show how to leverage largely commodity Ethernet switches to support the full aggregate bandwidth of clusters consisting of tens of thousands of elements. Similar to how clusters of commodity computers have largely replaced more specialized SMPs and MPPs, we argue that appropriately architected and interconnected commodity switches may deliver more performance at less cost than available from today's higher-end solutions. Our approach requires no modifications to the end host network interface, operating system, or applications; critically, it is fully backward compatible with Ethernet, IP, and TCP.

Subfields: Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/704ee1ed2c95bedd7808a92e879bd30cba818739

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

Citations in the corpus

Presto: Edge-based Load Balancing for Fast Datacenter Networks (2015, methodology): “Similar to previous works [2, 3, 54], our synthetic workloads include: Shufﬂe : Each server in the testbed sends 1GB data to every other server in the testbed in random order.”
Understanding network failures in data centers: measurement, analysis, and implications (2011, background): “This observation suggests that low-cost, commodity switches are not necessarily less reliable than their expensive, higher capacity counterparts and bodes well for data center networking proposals that focus on using commodity switches to build flat data center networks [3, 12, 21].”
A Survey on Network Methodologies for Real-Time Analytics of Massive IoT Data and Open Research Issues (2017, background): “The variations in these tree-based architectures are widely deployed in the data center networks of today [114]–[116].”
Less Is More: Trading a Little Bandwidth for Ultra-Low Latency in the Data Center (2012, methodology): “We simulate a three-layered fat-tree topology based on scalable data center architectures recently proposed [2, 22].”
Characterizing cloud computing hardware reliability (2010, background): “Further, if we develop a model that allows us to proactively predict failures, this can lead to moving workload and data off of such a server in time to avoid any possible service disruption.”
Design, Implementation and Evaluation of Congestion Control for Multipath TCP (2011, background): “FatTree simulations.”
On the impact of packet spraying in data center networks (2013, methodology): “ratios such as 4:1) such as the fat-tree architecture [4].”
Reining in the Outliers in Map-Reduce Clusters using Mantri (2010, background): “Though some recent proposals do away with capacity over-subscription in data centers [3, 18], today’s networks remain over-subscribed albeit with smaller ratios than those in the past.”
Orchestrating Virtualized Network Functions (2015, background): “28-port fat tree [27] with around 1000 nodes and 10K links as a data center network and an ISP network topology with 315 nodes and 972 links (AS1239 from RocketFuel ISP topologies [23]).”
Augmenting data center networks with multi-gigabit wireless links (2011, background): “When the trafﬁc matrix is sparse (i.e. only a few ToR switches are hot), a small number of ﬂyways can significantly improve performance, without the cost of building a fully non-oversubscribed network.”
pFabric: minimal near-optimal datacenter transport (2013, methodology): “Specifically, the datacenter fabric typically consists of two or three tiers of switches in a Fat-tree or Clos topology [1, 12].”
Planck (2014, methodology): “Even reacting in seconds can cause signiﬁcant disruption, so data center networks are frequently constructed to have full bisection bandwidth to avoid congestion [1, 13].”
Energy proportional datacenter networks (2010, background): “Even with a high-performance datacenter network based on a folded-Clos topology (i.e., fat trees) [2, 7, 17, 20] the network consumes only 12% of overall power at full utilization.”