Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks

By Yasar Sinan Nasir, Dongning Guo. IEEE Journal on Selected Areas in Communications, 2018

566 citations recorded by the source corpus.

This work demonstrates the potential of deep reinforcement learning techniques for transmit power control in wireless networks. Existing techniques typically find near-optimal power allocations by solving a challenging optimization problem. Most of these algorithms are not scalable to large networks in real-world scenarios because of their computational complexity and instantaneous cross-cell channel state information (CSI) requirement. In this paper, a distributively executed dynamic power allocation scheme is developed based on model-free deep reinforcement learning. Each transmitter collects CSI and quality of service (QoS) information from several neighbors and adapts its own transmit power accordingly. The objective is to maximize a weighted sum-rate utility function, which can be particularized to achieve maximum sum-rate or proportionally fair scheduling. Both random variations and delays in the CSI are inherently addressed using deep ${Q}$ -learning. For a typical network architecture, the proposed algorithm is shown to achieve near-optimal power allocation in real time based on delayed CSI measurements available to the agents. The proposed scheme is especially suitable for practical scenarios where the system model is inaccurate and CSI delay is non-negligible.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/82959dbab0aebb739967368f679970697cceb9a3

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

Citations in the corpus

Quantum Machine Learning for 6G Communication Networks: State-of-the-Art and Vision for the Future (2019, methodology): “In this regard, a model-free distributed reinforcement learning method for power allocation is proposed [79], in which Channel State Information (CSI) and QoS indicators are exploited to adapt the transmit power.”
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications (2021, methodology): “By considering CSI variations [275], [55], [276] and stochastic task arrivals [85], [84], the dynamic communication and computation resource allocation problem can be formulated as a MDP, for which deep RL, a model-free approach, can provide efﬁcient and robust solutions [73].”
Multi-Agent Reinforcement Learning Based Resource Management in MEC- and UAV-Assisted Vehicular Networks (2021, methodology): “existing works [23]–[26], we adopt RL methods to achieve real-time resource management in the considered scenario.”
Wireless Networks Design in the Era of Deep Learning: Model-Based, AI-Based, or Both? (2019, methodology): “The use of deep reinforcement learning is also considered in [196], where it is used to develop a power control algorithm for weighted sum-rate maximization in interference channels subject to maximum power constraints.”
Edge Learning for B5G Networks With Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing (2022, unknown): “In particular, a model-free DQL framework was proposed to apply the dynamic power allocation strategy for sum rate maximization with user fairness in a mobile Ad-hoc network [192].”
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges (2021, background): “work in [179] developed a novel MARL algorithm that enables”