Multiagent Learning: Basics, Challenges, and Prospects

By Karl Tuyls, Gerhard Weiss. 2012

219 citations recorded by the source corpus.

Multiagent systems (MAS) are widely accepted as an important method for solving problems of a distributed nature. A key to the success of MAS is efficient and effective multiagent learning (MAL). The past twenty-five years have seen a great interest and tremendous progress in the field of MAL. This article introduces and overviews this field by presenting its fundamentals, sketching its historical development and describing some key algorithms for MAL. Moreover, main challenges that the field is facing today are indentified.

Subfields: Artificial Intelligence, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

Citations in the corpus

Value-Decomposition Networks For Cooperative Multi-Agent Learning (2017, background): “…observability and aliasing (very small observation window combined with map symmetries), they still pose a serious challenge and is comparable to the state-of-the-art in multi-agent reinforcement learning (Leibo et al., 2017), which exceeds what is common in this area (Tuyls and Weiss, 2012).”
Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge? (2020, background): “…(MARL) has shown considerable promise in solving tasks that can be described as a Dec-POMDP [17], i.e., where agents optimise a single scalar team reward signal in a partially observable environment while choosing actions based only on their own local action-observation histories [18, 2, 29].”
Lenient Multi-Agent Deep Reinforcement Learning (2017, background): “One of the key challenges faced within MA-RL is the moving target problem: Given an environment with multiple agents whose rewards depend on each others’ actions, the difficulty of finding optimal policies for each agent is increased due to the policies of the agents being non stationary [5, 12, 31]”
A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning (2017, background): “Inspired by recent success in Deep RL, we are now seeing a renewed interest in multiagent reinforcement learning (MARL) [90, 17, 99].”
Emergent Communication through Negotiation (2018, background): “Recent work on deep multi-agent reinforcement learning (MARL) has shown great success in teaching agents complex behaviour without a complex environment simulator or demonstration data (Panait & Luke, 2005; Busoniu et al., 2008; Tuyls & Weiss, 2012; Silver et al., 2017; Leibo et al., 2017).”
Evolutionary Dynamics of Multi-Agent Learning: A Survey (2015, background): “SG networks of learning automata Vrancx et al. (2008a) Hennes et al.”
Neural Replicator Dynamics (2019, background): “The associated core challenge is that the agents’ concurrent learning implies that they each perceive the environment as nonstationary [37, 62].”
Value-Decomposition Networks For Cooperative Multi-Agent Learning Based On Team Reward (2018, background): “We consider the cooperative multi-agent reinforcement learning (MARL) problem [4, 17, 25], in which a system of several learning agents must jointly optimize a single reward signal – the team reward – accumulated over time.”
Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learning (2019, unknown): “As Lowe et al. (2017) has pointed out, in IL, the probability of taking a gradient step in the correct direction decreases exponentially with the increasing number of agents, letting alone the major issue of the non-stationary environment due to the independence assumption (Tuyls & Weiss, 2012).”
Multi-agent deep reinforcement learning: a survey (2021, background): “The marginalization approaches are closely related to the difference rewards proposed by Tumer and Wolpert (2004) who determine the impact of an agent’s individual action compared to the average reward of all agents.”
A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity (2017, background): “…to categorise MAS algorithms (Panait and Luke, 2005; Shoham et al., 2007; Busoniu et al., 2010), others have divided them by the type of learning (Tuyls and Weiss, 2012; Bloembergen et al., 2015) and another group have proposed properties that MAS algorithms should have (Bowling and Veloso,…”
Autonomous agents modelling other agents: A comprehensive survey and open problems (2017, background): “In addition to the above surveys, there are also a number of surveys on the topic of multiagent learning (Tuyls and Weiss, 2012; Busoniu et al., 2008; Panait and Luke, 2005; Alonso et al., 2001; Stone and Veloso, 2000; Sen and Weiss, 1999).”
Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence (2019, background): “Surveys of multi-agent intelligence research can be found here [13, 71, 84, 95].”
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms (2019, background): “As one of the most well-known issues in MARL, nonstationarity has long been recognized in the literature (Busoniu et al., 2008; Tuyls and Weiss, 2012).”