Deep Reinforcement Learning for User Association and Resource Allocation in Heterogeneous Cellular Networks

By Nan Zhao, Ying-Chang Liang, Dusist Niyato, Yiyang Pei, Minghu Wu, Yunhao Jiang. IEEE Transactions on Wireless Communications, 2019

385 citations recorded by the source corpus.

Heterogeneous cellular networks can offload the mobile traffic and reduce the deployment costs, which have been considered to be a promising technique in the next-generation wireless network. Due to the non-convex and combinatorial characteristics, it is challenging to obtain an optimal strategy for the joint user association and resource allocation issue. In this paper, a reinforcement learning (RL) approach is proposed to achieve the maximum long-term overall network utility while guaranteeing the quality of service requirements of user equipments (UEs) in the downlink of heterogeneous cellular networks. A distributed optimization method based on multi-agent RL is developed. Moreover, to solve the computationally expensive problem with the large action space, multi-agent deep RL method is proposed. Specifically, the state, action and reward function are defined for UEs, and dueling double deep Q-network (D3QN) strategy is introduced to obtain the nearly optimal policy. Through message passing, the distributed UEs can obtain the global state space with a small communication overhead. With the double-Q strategy and dueling architecture, D3QN can rapidly converge to a subgame perfect Nash equilibrium. Simulation results demonstrate that D3QN achieves the better performance than other RL approaches in solving large-scale learning problems.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/a7518e1cdee4aae6510eb0b79f4c66b9dd4b7081

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

Citations in the corpus

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey (2018, background): “networking. There are existing challenges, open issues, and new research directions which are discussed as follows. A. Challenges 1) State Determination in Density Networks: The DRL approaches, e.g., [23], allow the users to ﬁnd an optimal access policy without having complete and/or accurate networ”
Deep Reinforcement Learning for Task Offloading in Mobile Edge Computing Systems (2020, background): “Different from those works [17]–[20], we aim to propose a DRL-based distributed algorithm that can address the unknown load dynamics at the edge nodes.”