Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications

By Thanh Thi Nguyen, Ngoc Duy Nguyen, Saeid Nahavandi. 2018

1,032 citations recorded by the source corpus.

Reinforcement learning (RL) algorithms have been around for decades and employed to solve various sequential decision-making problems. These algorithms, however, have faced great challenges when dealing with high-dimensional environments. The recent development of deep learning has enabled RL methods to drive optimal policies for sophisticated and capable agents, which can perform efficiently in these challenging environments. This article addresses an important aspect of deep RL related to situations that require multiple agents to communicate and cooperate to solve complex tasks. A survey of different approaches to problems related to multiagent deep RL (MADRL) is presented, including nonstationarity, partial observability, continuous state and action spaces, multiagent training schemes, and multiagent transfer learning. The merits and demerits of the reviewed methods will be analyzed and discussed with their corresponding applications explored. It is envisaged that this review provides insights about various MADRL methods and can lead to the future development of more robust and highly useful multiagent learning methods for solving real-world problems.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Citations in the corpus

Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning (2019, background): “There has been a large number of follow-up papers in this area, so listing all relevant work is beyond the scope and we refer the reader to Nguyen et al. (2018), a recent survey on deep multi-agent RL.”
Multi-objective multi-agent decision making: a utility-based analysis and survey (2019, methodology): “For further information on recent multi-agent DRL methods, the interested reader is referred to recent survey articles [67,109].”
Learning scheduling algorithms for data processing clusters (2018, background): “To make the RL problem tractable, one potential research direction is to leverage multi-agent RL [38, 50, 62].”
Large Language Model Guided Tree-of-Thought (2023, methodology): “While there are many multi-agent reinforcement learning algorithms (MARL) proposed in the literature [29, 30, 31], in this work we adopt a relatively simple approach which uses a modiﬁed version of the REINFORCE algorithm [32] to train the policy networks of the ToT controller and the prompter…”
Artificial intelligence in the creative industries: a review (2020, background): “Temporal difference (TD) learning [35–37] has recently been introduced as a model-free reinforcement learning method that learns how to predict a quantity that depends on future values of a given signal.”
Deep Reinforcement Learning for Cyber Security (2019, methodology): “enable an effective large-scale defense plan. Challenges of multi-agent DRL itself then need to be addressed such as non-stationarity, partial observability, and efﬁcient multi-agent training schemes [183]. On the other hand, the RL methodology has been applied to deal with various cyber attacks, e.”
Deep Learning for Change Detection in Remote Sensing Images: Comprehensive Review and Meta-Analysis (2020, background): “Recently, deep reinforcement learning [111], [112] [113] has become the focus of considerable interest in the field of machine learning and has shown an excellent potential and great performance in various domains of computer vision such as autonomous driving [114], [115], object tracking [116], [11”
A review of cooperative multi-agent deep reinforcement learning (2019, background): “In [147], a review is provided for MARL, where the focus is on deep MARL from the following perspectives: non-stationarity, partial observability, continuous state and action spaces, training schemes, and transfer learning.”
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms (2019, background): “In fact, besides the classical reference [11], there are several other reviews on MARL that have appeared recently, due to the resurgence of MARL [40, 20, 41, 42].”
A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges (2023, neutral): “They actively observe the consequences of their actions, adjusting their behavior over time to maximize long-term cumulative rewards [42], [43].”
Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial (2020, methodology): “We refer the readers to [85], [87], and [88] for a extensive review of deep MARL literature.”
Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey (2020, background): “Reinforcement learning (RL) algorithms have been increasingly adopted by the robotics community over the past years to control complex robots or multi-robot systems [1], [2], or provide end-to-end policies from perception to control [3].”