An Overview of Recent Progress in the Study of Distributed Multi-Agent Coordination

By Yongcan Cao, Wenwu Yu, Wei Ren, Guanrong Chen. 2012

2,338 citations recorded by the source corpus.

This paper reviews some main results and progress in distributed multi-agent coordination, focusing on papers published in major control systems and robotics journals since 2006. Distributed coordination of multiple vehicles, including unmanned aerial vehicles, unmanned ground vehicles, and unmanned underwater vehicles, has been a very active research subject studied extensively by the systems and control community. The recent results in this area are categorized into several directions, such as consensus, formation control, optimization, and estimation. After the review, a short discussion section is included to summarize the existing research and to propose several promising research directions along with some open problems that are deemed important for further investigations.

Subfields: Aerospace Engineering, Computer Networks and Communications, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Networks and Communications

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2020, background): “Reinforcement learning (RL) holds considerable promise to help address a variety of cooperative multi-agent problems, such as coordination of robot swarms (H¨uttenrauch et al., 2017) and autonomous cars (Cao et al., 2012).”
Learning Multiagent Communication with Backpropagation (2016, background): “Th re i a rich liter ture on ulti-agent reinforc ent learning (MARL) [1], particularly in the robotics domain [18, 25, 5, 21, 2].”
Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge? (2020, methodology): “Many practical control problems feature a team of multiple agents that must coordinate to achieve a common goal [5, 12].”
QPLEX: Duplex Dueling Multi-Agent Q-Learning (2020, neutral): “, 2017), and autonomous cars (Cao et al., 2012).”
QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2018, background): “Reinforcement learning (RL) holds considerable promise to help address a variety of cooperative multi-agent problems, such as coordination of robot swarms (Hüttenrauch et al., 2017) and autonomous cars (Cao et al., 2012).”
Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks (2016, background): “Such problems arise naturally in a variety of settings, such as multi-robot systems and sensor networks (Matari, 1997; Fox et al., 2000; Gerkey & Matari, 2004; Olfati-Saber et al., 2007; Cao et al., 2013).”
MAVEN: Multi-Agent Variational Exploration (2019, methodology): “Cooperative multi-agent reinforcement learning (MARL) is a key tool for addressing many real-world problems such as coordination of robot swarms [22] and autonomous cars [6].”
Learning Nearly Decomposable Value Functions Via Communication Minimization (2019, background): “Cooperative multi-agent reinforcement learning (MARL) are ﬁnding applications in many real-world domains, such as autonomous vehicle teams (Cao et al., 2012), intelligent warehouse systems (Now ´ e et al., 2012), and sensor networks (Zhang & Lesser, 2011).”
Deep Multi-Agent Reinforcement Learning with Discrete-Continuous Hybrid Action Spaces (2019, background): “Reinforcement learning (RL) has recently shown a great success on a variety of cooperative multi-agent problems, such as multiplayer games [Peng et al. , 2017], autonomous cars [Cao et al. , 2013] and network packet delivery [Ye et al. , 2015].”
Multi-Agent Common Knowledge Reinforcement Learning (2018, background): “oof-of-concept matrix game and a multi-agent version of StarCraft II Micromanagement. Introduction Cooperative multi-agent systems are ubiquitous, for example, in the coordination of autonomous cars (Cao et al., 2013). However, how to learn control policies for such systems remains a major open ques”
Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments (2017, background): “nce paper at ICLR 2018 due to complexity of the emergent behavior and are of practical interest with applications ranging from multiplayer games (Peng et al., 2017) to coordinating self-driving ﬂeets Cao et al. (2013). Multi-agent environments are nonstationary from the perspective of any individual”
On the Convergence of Decentralized Gradient Descent (2013, background): “These properties are important for applications where data are collected by distributed agents, communication to a fusion center is expensive or impossible, and/or agents tend to keep their raw data private; such applications arise in wireless sensor networks [16, 24, 27, 38], multivehicle and multi”
Learning Attentional Communication for Multi-Agent Cooperation (2018, background): “Similarly, communication can be crucially important in multi-agent reinforcement learning (MARL) for cooperation, especially for the scenarios where a large number of agents work in a collaborative way, such as autonomous vehicles planning [1], smart grid control [20], and multi-robot control [14].”
DSA: Decentralized Double Stochastic Averaging Gradient Algorithm (2015, background): “Deﬁne x n ∈ R p as a local copy of the variable x that is kept at node n .”
ROMA: Multi-Agent Reinforcement Learning with Emergent Roles (2020, background): “Many real-world systems can be modeled as multi-agent systems (MAS), such as autonomous vehicle teams (Cao et al., 2012), intelligent warehouse systems (Now´e et al., 2012), and sensor networks (Zhang & Lesser, 2011).”
An Exact Quantized Decentralized Gradient Descent Algorithm (2018, background): “D ISTRIBUTED optimization of a sum of convex functions has a variety of applications in different areas including decentralized control systems [2], wireless systems [3], sensor networks [4], networked multiagent systems [5], multirobot networks [6], and large scale machine learning [7].”
Compressed Distributed Gradient Descent: Communication-Efficient Consensus over Networks (2018, background): “, distributed sensing in wireless sensor networks [1]–[4], decentralized machine learning [5], [6], multi-agent robotic systems [7]–[9], smart grids [10], [11], to name just a few.”