Multi-Agent Reinforcement Learning-Based Resource Allocation for UAV Networks

By Jingjing Cui, Yuanwei Liu, Arumgam Nallanathan. IEEE Transactions on Wireless Communications, 2018

539 citations recorded by the source corpus.

Unmanned aerial vehicles (UAVs) are capable of serving as aerial base stations (BSs) for providing both cost-effective and on-demand wireless communications. This article investigates dynamic resource allocation of multiple UAVs enabled communication networks with the goal of maximizing long-term rewards. More particularly, each UAV communicates with a ground user by automatically selecting its communicating user, power level and subchannel without any information exchange among UAVs. To model the dynamics and uncertainty in environments, we formulate the long-term resource allocation problem as a stochastic game for maximizing the expected rewards, where each UAV becomes a learning agent and each resource allocation solution corresponds to an action taken by the UAVs. Afterwards, we develop a multi-agent reinforcement learning (MARL) framework that each agent discovers its best strategy according to its local observations using learning. More specifically, we propose an agent-independent method, for which all agents conduct a decision algorithm independently but share a common structure based on Q-learning. Finally, simulation results reveal that: 1) appropriate parameters for exploitation and exploration are capable of enhancing the performance of the proposed MARL based resource allocation algorithm; 2) the proposed MARL algorithm provides acceptable performance compared to the case with complete information exchanges among UAVs. By doing so, it strikes a good tradeoff between performance gains and information exchange overheads.

Subfields: Aerospace Engineering, Computer Networks and Communications, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/1f9bb9e2edb6c8721e85c0c1141ddb45c9d9fb99

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Networks and Communications

In Electrical and Electronic Engineering

Citations in the corpus

Edge Learning for B5G Networks With Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing (2022, unknown): “For example in [87], an Independent Learner”
Artificial Neural Networks-Based Machine Learning for Wireless Networks: A Tutorial (2017, methodology): “In [95], an ESN based RL algorithm is proposed for resource allocation in UAV based networks.”
Multi-Agent Reinforcement Learning: A Review of Challenges and Applications (2021, methodology): “Another UAV application used a UAV team to implement a network for line-of-sight (LoS) communications using MARL to optimize resource management, including the transmit power and subchannel selection [50].”
Multi-Agent Reinforcement Learning Based Resource Management in MEC- and UAV-Assisted Vehicular Networks (2021, background): “And the average channel gain between vehicle i and UAV j ∈ U at time slot t , G i,j ( t ) , is deﬁned similar to [28].”