Interference Management for Cellular-Connected UAVs: A Deep Reinforcement Learning Approach

By Ursula Challita, Walid Saad, Christian Bettstetter. IEEE Transactions on Wireless Communications, 2018

336 citations recorded by the source corpus.

In this paper, an interference-aware path planning scheme for a network of cellular-connected unmanned aerial vehicles (UAVs) is proposed. In particular, each UAV aims at achieving a tradeoff between maximizing energy efficiency and minimizing both wireless latency and the interference caused on the ground network along its path. The problem is cast as a dynamic game among UAVs. To solve this game, a deep reinforcement learning algorithm, based on echo state network (ESN) cells, is proposed. The introduced deep ESN architecture is trained to allow each UAV to map each observation of the network state to an action, with the goal of minimizing a sequence of time-dependent utility functions. Each UAV uses the ESN to learn its optimal path, transmission power, and cell association vector at different locations along its path. The proposed algorithm is shown to reach a subgame perfect Nash equilibrium upon convergence. Moreover, an upper bound and a lower bound for the altitude of the UAVs are derived thus reducing the computational complexity of the proposed algorithm. The simulation results show that the proposed scheme achieves better wireless latency per UAV and rate per ground user (UE) while requiring a number of steps that are comparable to a heuristic baseline that considers moving via the shortest distance toward the corresponding destinations. The results also show that the optimal altitude of the UAVs varies based on the ground network density and the UE data rate requirements and plays a vital role in minimizing the interference level on the ground UEs as well as the wireless transmission delay of the UAV.

Subfields: Aerospace Engineering, Computer Networks and Communications, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/1a0567770dff874efd92917a32662f8c0cfc1c58

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Networks and Communications

In Electrical and Electronic Engineering

Citations in the corpus

Multi-Agent Reinforcement Learning-Based Resource Allocation for UAV Networks (2018, background): “In [22], the authors studied the autonomous path planning of UAVs by jointly taking energy efficiency, latency and interference into consideration, in which an echo state networks based deep reinforcement learning algorithm was proposed.”
Evolution of Non-Terrestrial Networks From 5G to 6G: A Survey (2021, background): “Optimal path planning of multiple cellularconnected UAVs is considered in [225].”
Accessing From the Sky: A Tutorial on UAV Communications for 5G and Beyond (2019, methodology): “A noncooperative game formulation was given in [230].”