Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access

By Oshri Naparstek, Kobi Cohen. IEEE Transactions on Wireless Communications, 2017

431 citations recorded by the source corpus.

We consider the problem of dynamic spectrum access for network utility maximization in multichannel wireless networks. The shared bandwidth is divided into $K$ orthogonal channels. In the beginning of each time slot, each user selects a channel and transmits a packet with a certain transmission probability. After each time slot, each user that has transmitted a packet receives a local observation indicating whether its packet was successfully delivered or not (i.e., ACK signal). The objective is a multi-user strategy for accessing the spectrum that maximizes a certain network utility in a distributed manner without online coordination or message exchanges between users. Obtaining an optimal solution for the spectrum access problem is computationally expensive, in general, due to the large-state space and partial observability of the states. To tackle this problem, we develop a novel distributed dynamic spectrum access algorithm based on deep multi-user reinforcement leaning. Specifically, at each time slot, each user maps its current state to the spectrum access actions based on a trained deep-Q network used to maximize the objective function. Game theoretic analysis of the system dynamics is developed for establishing design principles for the implementation of the algorithm. The experimental results demonstrate the strong performance of the algorithm.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/bd75ef44d2a706d02fa99b1c1e5d293d30a411ec

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

1. Ying-Chang Liang
2. Yiyang Pei
3. Fei Hu
4. Qian Mao
5. Dusist Niyato
6. Qi Hao
7. Dongning Guo
8. Yasar Sinan Nasir

In Electrical and Electronic Engineering

1. Ying-Chang Liang
2. Dusist Niyato
3. Yiyang Pei
4. Fei Hu
5. Qian Mao
6. Qi Hao
7. Dongning Guo
8. Yasar Sinan Nasir

Citations in the corpus

• Deep Learning for Intelligent Wireless Networks: A Comprehensive Survey (2018, methodology): “networks with less computation and limited observations, Naparstek and Cohen [60] proposed a deep multi-user rein-”
• Deep Reinforcement Learning for User Association and Resource Allocation in Heterogeneous Cellular Networks (2019, methodology): “Furthermore, multi-agent DRL method has also be considered in the above related areas [35], [36].”
• Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks (2018, background): “1) Many states are rarely visited, and 2) the storage of lookup table becomes impractical [28].”