TCP ex machina: computer-generated congestion control

By Keith Winstein, Hari Balakrishnan. 2013

581 citations recorded by the source corpus.

This paper describes a new approach to end-to-end congestion control on a multi-user network. Rather than manually formulate each endpoint's reaction to congestion signals, as in traditional protocols, we developed a program called Remy that generates congestion-control algorithms to run at the endpoints. In this approach, the protocol designer specifies their prior knowledge or assumptions about the network and an objective that the algorithm will try to achieve, e.g., high throughput and low queueing delay. Remy then produces a distributed algorithm---the control rules for the independent endpoints---that tries to achieve this objective. In simulations with ns-2, Remy-generated algorithms outperformed human-designed end-to-end techniques, including TCP Cubic, Compound, and Vegas. In many cases, Remy's algorithms also outperformed methods that require intrusive in-network changes, including XCP and Cubic-over-sfqCoDel (stochastic fair queueing with CoDel for active queue management). Remy can generate algorithms both for networks where some parameters are known tightly a priori, e.g. datacenters, and for networks where prior knowledge is less precise, such as cellular networks. We characterize the sensitivity of the resulting performance to the specificity of the prior knowledge, and the consequences when real-world conditions contradict the assumptions supplied at design-time.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

Citations in the corpus

Certified Defenses against Adversarial Examples (2018, background): “More broadly, certiﬁcates of performance are likely necessary for deploying machine learning systems in critical infrastructure such as internet packet routing (Winstein & Balakrishnan, 2013; Sivaraman et al., 2014).”
A comprehensive survey on machine learning for networking: evolution, applications and research opportunities (2018, background): “Remy [477] Own (offline training) · Wired · Cellular ns-2 simulation: · Wired topology: - Dumbbell - Datacenter · Cellular topology States: · IAT of ACKs · IAT of packets sent · RTT Reward: · Throughput · Delay Continuous with 3-dimensions: · CWNDmultiple · CWND increment · Time between successive s”
Experience-driven Networking: A Deep Reinforcement Learning based Approach (2018, methodology): “In order to address both throughput and delay, similar as in [36], we define a utility function U(·) for session k:”
Oboe: auto-tuning video ABR algorithms to network conditions (2018, methodology): “[56] used simulations to determine TCP parameters for different settings, while Semke et al.”
PCC Vivace: Online-Learning Congestion Control (2018, background): “While they can attain high performance, Remy-generated TCPs are inherently prone to degraded performance when the actual network conditions deviate from input assumptions [27].”
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey (2018, unknown): “based on end-to-end throughput and delay [145].”
Resource Management with Deep Reinforcement Learning (2016, methodology): “Recently, learning has been applied to designing congestion control protocols using a large number of ofﬂine [37] or on-line [13] experiments.”
Darwin or Lamarck? Future Challenges in Evolutionary Algorithms for Knowledge Discovery and Data Mining (2014, methodology): “Evolutionary methods are used in many different research fields such as medicine [14], genetics [15], or engineering [16], and there are nearly countless application areas of EAs, due to their adaptive nature and ability in solving difficult optimization problems [17], [18], [19].”
A Concise Introduction to Decentralized POMDPs (2016, methodology): “Its actions consist of adjustments to the congestion window (increments and multipliers) and the minimum interval between outgoing packets [Winstein and Balakrishnan, 2013].”
Copa: Practical Delay-Based Congestion Control for the Internet (2018, background): “In many scenarios these objective-optimization meth-ods outperform the more traditional window-update schemes [9, 43].”
Learning in situ: a randomized experiment in video streaming (2019, background): “Results of primary experiment (Jan. 26 work [25,37,44].”
Machine Learning for Networking: Workflow, Advances and Opportunities (2017, methodology): “In addition, it has been proved in the machine learning domain that learning with a simulator rather than in a real environment is more effective and with lower cost in RL scenarios [3].”
A Deep Reinforcement Learning Perspective on Internet Congestion Control (2019, methodology): “Thus, unlike RL, Remy does not learn to react to patterns in traffic and network conditions, but relies on human-specified assumptions.”