Deep Multi-User Reinforcement Learning for Dynamic Spectrum Access in Multichannel Wireless Networks

By Oshri Naparstek, Kobi Cohen. 2017

104 citations recorded by the source corpus.

We consider the problem of dynamic spectrum access for network utility maximization in multichannel wireless networks. The shared bandwidth is divided into K orthogonal channels, and the users access the spectrum using a random access protocol. In the beginning of each time slot, each user selects a channel and transmits a packet with a certain attempt probability. After each time slot, each user that has transmitted a packet receives a local observation indicating whether its packet was successfully delivered or not (i.e., ACK signal). The objective is to find a multi-user strategy that maximizes a certain network utility in a distributed manner without online coordination or message exchanges between users. Obtaining an optimal solution for the spectrum access problem is computationally expensive in general due to the large state space and partial observability of the states. To tackle this problem, we develop a distributed dynamic spectrum access algorithm based on deep multi-user reinforcement leaning. Specifically, at each time slot, each user maps its current state to spectrum access actions based on a trained deep-Q network used to maximize the objective function. Experimental results have demonstrated that users are capable to learn good policies that achieve strong performance in this challenging partially observable setting only from their ACK signals, without online coordination, message exchanges between users, or carrier sensing.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

Citations in the corpus

Wireless Network Intelligence at the Edge (2018, background): “The effective amount of bandwidth can also be increased using low-complexity dynamic spectrum access [44] and proactive resource management [45].”
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey (2018, background): “As proved in [27], the game has a subgame perfect”
Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks (2018, methodology): “em at hand. As problem size increases, the classical Q-learning algorithm fails because of two main reasons: many states are rarely visited and the storage of lookup table in (20) becomes impractical [23]. These issues can be solved with deep reinforcement learning, e.g., deep Q-learning [11]. A dee”
Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks (2017, methodology): “The MAC scheme in [13] employs DRL in homogeneous wireless networks.”
Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access (2017, methodology): “A short version of this paper that introduces the algorithm and preliminary experimental results was presented at the IEEE Global Communications Conference (GLOBECOM), 2017 [1].”
Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks (2019, background): “In order to curb the potentially excessive computational complexity resulting from having a large state space and to deal with its partial observability in cognitive radio networks, Naparstek and Cohen developed a distributed dynamic spectrum access scheme relying on deep multi-user reinforcement le”
Deep Learning in Mobile and Wireless Networking: A Survey (2018, background): “Radio Control: In [306], the authors address the dynamic spectrum access problem in multichannel wireless network environments using deep reinforcement learning.”