Experience-driven Networking: A Deep Reinforcement Learning based Approach

By Zhiyuan Xu, Jian Tang, Jingsong Meng, Weiyi Zhang, Yanzhi Wang, Chi Harold Liu, Dejun Yang. IEEE Conference on Computer Communications, 2018

414 citations recorded by the source corpus.

Modern communication networks have become very complicated and highly dynamic, which makes them hard to model, predict and control. In this paper, we develop a novel experience-driven approach that can learn to well control a communication network from its own experience rather than an accurate mathematical model, just as a human learns a new skill (such as driving, swimming, etc). Specifically, we, for the first time, propose to leverage emerging Deep Reinforcement Learning (DRL) for enabling model-free control in communication networks; and present a novel and highly effective DRL-based control framework, DRL-TE, for a fundamental networking problem: Traffic Engineering (TE). The proposed framework maximizes a widely-used utility function by jointly learning network environment and its dynamics, and making decisions under the guidance of powerful Deep Neural Networks (DNNs). We propose two new techniques, TE-aware exploration and actor-critic-based prioritized experience replay, to optimize the general DRL framework particularly for TE. To validate and evaluate the proposed framework, we implemented it in ns-3, and tested it comprehensively with both representative and randomly generated network topologies. Extensive packet-level simulation results show that 1) compared to several widely-used baseline methods, DRL-TE significantly reduces end-to-end delay and consistently improves the network utility, while offering better or comparable throughput; 2) DRL-TE is robust to network changes; and 3) DRL-TE consistently outperforms a state-of-the-art DRL method (for continuous control), Deep Deterministic Policy Gradient (DDPG), which, however, does not offer satisfying performance.

Subfields: Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/ec1d30b1fba63149f0ac63d3094585172caeb864

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

Citations in the corpus

Deep Learning in Mobile and Wireless Networking: A Survey (2018, methodology): “[390] employ a DRL agent to learn from network dynamics how to control traffic flow.”
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey (2018, methodology): “Experimental results show that the conventional DDPG method does not work well for the continuous control problem in [144].”
RouteNet: Leveraging Graph Neural Networks for Network Modeling and Optimization in SDN (2019, background): “Relevant research efforts are being devoted to apply neural networks to model computer networks [5] and using such models for network optimization [6], [7], [3].”