Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks

By Shangxing Wang, Hanpeng Liu, Pedro Henrique de Oliveira Gomes, Bhaskar Krishnamachari. IEEE Transactions on Cognitive Communications and Networking, 2018

415 citations recorded by the source corpus.

We consider a dynamic multichannel access problem, where multiple correlated channels follow an unknown joint Markov model and users select the channel to transmit data. The objective is to find a policy that maximizes the expected long-term number of successful transmissions. The problem is formulated as a partially observable Markov decision process with unknown system dynamics. To overcome the challenges of unknown dynamics and prohibitive computation, we apply the concept of reinforcement learning and implement a deep Q-network (DQN). We first study the optimal policy for fixed-pattern channel switching with known system dynamics and show through simulations that DQN can achieve the same optimal performance without knowing the system statistics. We then compare the performance of DQN with a Myopic policy and a Whittle Index-based heuristic through both more general simulations as well as real data trace and show that DQN achieves near-optimal performance in more complex situations. Finally, we propose an adaptive DQN approach with the capability to adapt its learning in time-varying scenarios.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/3b4c398f234e946d599b492d8a4bba4eb8376122

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

Citations in the corpus

Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial (2020, methodology): “As an example, [45] solves the dynamic multichannel access problem in wireless networks using DQN.”
Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks (2017, methodology): “etween [15] and our investigation here. The Markov environmental model in [15] cannot capture the interactions among nodes due to their MAC protocols. In particular, the Markov environmental model in [15] is a “passive” model not affected by the “actions” of the DRL agent. For example, if there is o”
Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks (2019, methodology): “[353] proposed an adaptive DQN algorithm for dynamic multichannel access, which was capable of achieving a near-optimal performance outperforming the Myopic policy [354]8 and the Whittle’s Index-based heuristic algorithms [355]9 in complex scenarios.”
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey (2018, background): “An adaptive DQL scheme is proposed in [48] which evaluates the accumulated reward of the current policy for every period.”
Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access (2017, methodology): “In [12], [13], the authors developed a spectrum sensing policy based on DRL for a single user who interacts with an external environment.”
Deep Reinforcement Learning for User Association and Resource Allocation in Heterogeneous Cellular Networks (2019, methodology): “Recently, DRL-based approach has been applied to several research fields, such as resource management and allocation [25]–[31], mobile offloading [32], dynamic channel access [33], fog radio access networks [30], mobile edge computing and caching [34].”