Cooperative and Distributed Reinforcement Learning of Drones for Field Coverage

By Huy Xuan Pham, Hung Manh La, David Feil-Seifer, Luan Van Nguyen. 2018

105 citations recorded by the source corpus.

This paper proposed a distributed Multi-Agent Reinforcement Learning (MARL) algorithm for a team of Unmanned Aerial Vehicles (UAVs) that can learn to cooperate to provide a full coverage of an unknown field of interest while minimizing the overlapping sections among their field of views. Two challenges in MARL for such a system are discussed in the paper: firstly, the complex dynamic of the joint-actions of the UAV team, that will be solved using game-theoretic correlated equilibrium, and secondly, the challenge in huge dimensional state space will be tackled with an efficient space-reduced representation of the value function table. We also provided our experimental results in details with both simulation and physical implementation to show that the UAVs can successfully learn to accomplish the task without the need of a mathematical model. This paper can serve as an efficient framework for using MARL to enable UAV team to work in an environment where its model is unavailable. In the future, we are interested in accounting for the stochastic aspect of the problem and technical aspects of a real-world deployment, where uncertainties, such as wind and other dynamics of the environment presents.

Subfields: Aerospace Engineering, Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

RODE: Learning Roles to Decompose Multi-Agent Tasks (2020, background): “Cooperative multi-agent problems are ubiquitous in real-world applications, such as crewless aerial vehicles (Pham et al., 2018; Xu et al., 2018) and sensor networks (Zhang & Lesser, 2013).”
Multi-Agent Common Knowledge Reinforcement Learning (2018, neutral): “, 2013) or unmanned aerial vehicles (Pham et al., 2018; Xu et al., 2018).”
Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial (2020, background): “The problem of field coverage by a team of UAVs is addressed in [27].”
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms (2019, methodology): “Here we choose unmanned aerial vehicles (UAVs) [334, 335, 336, 337, 338, 339], a recently surging application scenario of multi-agent autonomous systems, as one representative example.”
Multi-Agent Reinforcement Learning: A Review of Challenges and Applications (2021, methodology): “In [47], a team of UAVs equipped with a downward-facing camera was considered; the UAVs were assigned the task of optimal sensing coverage.”
Multi-vehicle Flocking Control with Deep Deterministic Policy Gradient Method (2018, background): “cn Recent development of the reinforcement learning (RL) [7] has provided an alternative way to deal with the vehicle control problem and shows its potential on applications that involve interaction between multiple vehicles, such as the collision avoidance [8], communication [9], and environment ex”