Coordinated Multi-Robot Exploration Under Communication Constraints Using Decentralized Markov Decision Processes

By Laëtitia Matignon, Laurent Jeanpierre, Abdel-Illah Mouaddib. 2012

151 citations recorded by the source corpus.

Recent works on multi-agent sequential decision making using decentralized partially observable Markov decision processes have been concerned with interaction-oriented resolution techniques and provide promising results. These techniques take advantage of local interactions and coordination. In this paper, we propose an approach based on an interaction-oriented resolution of decentralized decision makers. To this end, distributed value functions (DVF) have been used by decoupling the multi-agent problem into a set of individual agent problems. However existing DVF techniques assume permanent and free communication between the agents. In this paper, we extend the DVF methodology to address full local observability, limited share of information and communication breaks. We apply our new DVF in a real-world application consisting of multi-robot exploration where each robot computes locally a strategy that minimizes the interactions between the robots and maximizes the space coverage of the team even under communication constraints. Our technique has been implemented and evaluated in simulation and in real-world scenarios during a robotic challenge for the exploration and mapping of an unknown environment. Experimental results from real-world scenarios and from the challenge are given where our system was vice-champion.

Subfields: Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments (2017, background): “re a number of important applications that involve interaction between multiple agents, where emergent behavior and complexity arise from agents co-evolving together. For example, multi-robot control [21], the discovery of communication and language [31, 8, 25], multiplayer games [28], and the analy”
Learning Attentional Communication for Multi-Agent Cooperation (2018, background): “Similarly, communication can be crucially important in multi-agent reinforcement learning (MARL) for cooperation, especially for the scenarios where a large number of agents work in a collaborative way, such as autonomous vehicles planning [1], smart grid control [20], and multi-robot control [14].”
Multi-agent deep reinforcement learning: a survey (2021, background): “One can either ignore the existence of other agents (Matignon et al. 2012b) or discount the adaptive behavior by assuming the others’ behavior to be static or optimal (Lauer and Riedmiller 2000).”
Multi-Agent Generative Adversarial Imitation Learning (2018, background): “This problem is further exacerbated in multi-agent scenarios, such as multiplayer games (Peng et al., 2017), multi-robot control (Matignon et al., 2012) and social interactions (Leibo et al., 2017); in these cases, agents do not even necessarily share the same reward function, especially in…”
Multi-Agent Actor-Critic with Hierarchical Graph Attention Network (2019, unknown): “…agents are also important modeling paradigms in various engineering systems, such as smart grids (Dall’Anese, Zhu, and Giannakis 2013), logistics (Ying and Dayong 2005; Cao et al. 2013), and distributed vehicles/robots (Corke, Peterson, and Rus 2005; Fax and Murray 2004; Matignon et al. 2012).”
Graph Neural Networks for Decentralized Multi-Robot Path Planning (2019, background): “restricted in observation and communication range. This naturally arises when considering physical robots equipped with hardware constraints that limit their perception and communication capabilities [16]. These scenarios impose a decentralized structure, where at any given point in time, robots hav”
Graph Convolutional Reinforcement Learning (2018, background): “It is crucially important to enable agents to learn to cooperate in multi-agent environments for many applications, e.g. , autonomous driving (Shalev-Shwartz et al., 2016), trafﬁc light control (Wiering, 2000), smart grid control (Yang et al., 2018a), and multi-robot control (Matignon et al., 2012).”
Graph Convolutional Reinforcement Learning for Multi-Agent Cooperation (2018, background): “It is crucially important to enable agent to learn to cooperate in multi-agent environments for many applications, e.g., autonomous driving (Shalev-Shwartz et al., 2016), traffic light control (Wiering, 2000), smart grid control (Yang et al., 2018a), and multi-robot control (Matignon et al., 2012).”
Learning Efficient Communication in Cooperative Multi-Agent Environment (2019, background): “In multi-agent reinforcement learning (MARL) collaboration, communication is critical, especially for scenarios where a large number of agents work collaboratively, such as autonomous vehicles planning [3], smart grid control [12], and multi-robot control [10].”