Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

By Nguyen Cong Luong, Dinh Thai Hoang, Shimin Gong, Dusist Niyato, Ping Wang, Ying-Chang Liang, Dong In Kim. IEEE Communications Surveys and Tutorials, 2018

1,781 citations recorded by the source corpus.

This paper presents a comprehensive literature review on applications of deep reinforcement learning (DRL) in communications and networking. Modern networks, e.g., Internet of Things (IoT) and unmanned aerial vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, DRL, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of DRL from fundamental concepts to advanced models. Then, we review DRL approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks, such as 5G and beyond. Furthermore, we present applications of DRL for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying DRL.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/79c8f930bb66c82421b84617e4b6c0b2855cd063

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

Citations in the corpus

Distributed Learning in Wireless Networks: Recent Progress and Future Challenges (2021, background): “All of the previous works in [172], [175]–[182] are focused on the design of novel RL algorithms for solving wireless communication problems and they did not consider the issues of implementing RL algorithms over resource constrained wireless networks.”
Machine Learning for Wireless Networks with Artificial Intelligence: A Tutorial on Neural Networks (2017, methodology): “SON Multi-RAT IoT PL CR DT [3] p p p [32] p p [50] p [51] p p p p [52] p p p [53] p p p p [54] p p p [63] p [55] p p [56] p p p p [57] p p p p p p [58] p p p p p p [59] p p p p p [60] p p p p p p p p [61] p p p p p p p p p [62] p p p p p p p Our tutorial p p p p p p p p p p p p Section I: Introducti”
Evolution of NOMA Toward Next Generation Multiple Access (NGMA) for 6G (2021, background): “Reinforcement learning (RL), which empowers the models to learn from the dynamic/uncertain environment and to learn from their historical experiences, is capable of overcoming these challenges and provides an efficient approach for tackling resource allocation problems in dynamic heterogeneous netwo”
A Survey of Multi-Access Edge Computing in 5G and Beyond: Fundamentals, Technology Integration, and State-of-the-Art (2019, neutral): “Moreover, some surveys and tutorials on ML, DL, (deep) RL, as well as their applications in communications and networking [230]–[232] have come out, and readers can refer to these literature for more details.”
A Review of Deep Reinforcement Learning for Smart Building Energy Management (2020, background): “communications and networking, autonomous driving, autonomous IoT, cyber security, and multi-agent systems can be found in [24], [33]–[36], [52]–[56].”
Multi-agent deep reinforcement learning: a survey (2021, background): “…of distributed systems (Cao et al. 2013; Wang et al. 2016b) such as autonomous vehicles (Shalev-Shwartz et al. 2016) and multi-robot control (Matignon et al. 2012a), the networking of communication packages (Luong et al. 2019), or the trading on financial markets (Lux and Marchesi 1999).”
The Roadmap to 6G - AI Empowered Wireless Networks (2019, methodology): “For example, [11] recently applied DRL to address several emerging issues in communication and networking, including adaptive modulation, wireless caching, data ofﬂoading, and so on, as shown in Fig.”
Deep Learning in Mobile and Wireless Networking: A Survey (2018, background): “[71] A survey of deep reinforcement learning for networking.”
Artificial intelligence based anomaly detection of energy consumption in buildings: A review, current trends and new perspectives (2021, background): “broad variety of issues, including detecting abnormalities video surveillance, traffic management and anomaly detection [244, 245], communication and networking [246] and energy consumption prediction [247].”
Reconfigurable Intelligent Surfaces: Principles and Opportunities (2020, background): “Figure 15: Key features, pros and cons of reinforcement learning algorithms [116]–[126].”
A Survey of Rate-Optimal Power Domain NOMA With Enabling Technologies of Future Wireless Networks (2019, background): “communications systems where ad-hoc, autonomous decisions need to be taken [152].”
Federated Learning in Mobile Edge Networks: A Comprehensive Survey (2019, background): “Similarly, [30] considers the use of DRL in communications”
Survey on 6G Frontiers: Trends, Applications, Requirements, Technologies and Future Research (2021, background): “Many AI frameworks, including are machine learning [53], deep reinforcement learning [217], deep learning, distributed learning [68], FL [213], and computational intelligence [221], have been investigated to enable hyperintelligent networking such as AI-driven air interface, intelligent orchestratio”
A Vision and Framework for the High Altitude Platform Station (HAPS) Networks of the Future (2020, background): “of reinforcement learning, “deep reinforcement learning,” has emerged, which applies the intelligent data representation of the deep neural network in the reinforcement learning [244].”
Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial (2017, background): “First, a majority of the existing works focuses on a single ML technique (often the basics of deep learning [32], [50], and [56]–[58] or reinforcement learning [61]) and, as such, they do not capture the rich spectrum of available ML frameworks.”
Machine Learning in the Air (2019, positive): “In either case, the problem could then be formulated as a reinforcement learning (RL) problem [73], which has been successfully applied to many communication problems [74].”