Comparative evaluation of MAL algorithms in a diverse set of ad hoc team problems
By Stefano V. Albrecht, Subramanian Ramamoorthy. 2012
32 citations recorded by the source corpus.
This paper is concerned with evaluating different multiagent learning (MAL) algorithms in problems where individual agents may be heterogenous, in the sense of utilizing different learning strategies, without the opportunity for prior agreements or information regarding coordination. Such a situation arises in ad hoc team problems, a model of many practical multiagent systems applications. Prior work in multiagent learning has often been focussed on homogeneous groups of agents, meaning that all agents were identical and a priori aware of this fact. Also, those algorithms that are specifically designed for ad hoc team problems are typically evaluated in teams of agents with fixed behaviours, as opposed to agents which are adapting their behaviours. In this work, we empirically evaluate five MAL algorithms, representing major approaches to multiagent learning but originally developed with the homogeneous setting in mind, to understand their behaviour in a set of ad hoc team problems. All teams consist of agents which are continuously adapting their behaviours. The algorithms are evaluated with respect to a comprehensive characterisation of repeated matrix games, using performance criteria that include considerations such as attainment of equilibrium, social welfare and fairness. Our main conclusion is that there is no clear winner. However, the comparative evaluation also highlights the relative strengths of different algorithms with respect to the type of performance criteria, e.g., social welfare vs. attainment of equilibrium.
Subfields: Artificial Intelligence, Computer Networks and Communications
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Stefano V. Albrecht
- Filippos Christianos
- Georgios Papoudakis
- Arrasy Rahman
- Peter Stone
- Lukas Schäfer
- Subramanian Ramamoorthy
- Matthew E. Taylor
In Computer Networks and Communications
Citations in the corpus
- Autonomous agents modelling other agents: A comprehensive survey and open problems (2017, background): “[9,49,133]) and learning based on communication.”
- Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning (2019, background): “Furthermore, research on how agents can effectively deal with heterogeneity in capabilities and learning algorithms (Albrecht and Ramamoorthy, 2012) will play an important part.”
- Comparative Evaluation of Multi-Agent Deep Reinforcement Learning Algorithms (2020, background): “The only exception is the study of Albrecht and Ramamoorthy [2] which however focused on classic (non-deep) MARL algorithms and simple matrix games.”
- A game-theoretic model and best-response learning method for ad hoc coordination in multiagent systems (2013, methodology): “On the other hand, much work in multiagent systems has focused on efficiency, whilst often making central assumptions about the other agent’s behaviours [Albrecht and Ramamoorthy, 2012].”