Multi-objective multi-agent decision making: a utility-based analysis and survey

By Roxana Rădulescu, Patrick Mannion, Diederik Marijn Roijers, Ann Nowé. 2019

188 citations recorded by the source corpus.

The majority of multi-agent system implementations aim to optimise agents’ policies with respect to a single objective, despite the fact that many real-world problem domains are inherently multi-objective in nature. Multi-objective multi-agent systems (MOMAS) explicitly consider the possible trade-offs between conflicting objective functions. We argue that, in MOMAS, such compromises should be analysed on the basis of the utility that these compromises have for the users of a system. As is standard in multi-objective optimisation, we model the user utility using utility functions that map value or return vectors to scalar values. This approach naturally leads to two different optimisation criteria: expected scalarised returns (ESR) and scalarised expected returns (SER). We develop a new taxonomy which classifies multi-objective multi-agent decision making settings, on the basis of the reward structures, and which and how utility functions are applied. This allows us to offer a structured view of the field, to clearly delineate the current state-of-the-art in multi-objective multi-agent decision making approaches and to identify promising directions for future research. Starting from the execution phase, in which the selected policies are applied and the utility for the users is attained, we analyse which solution concepts apply to the different settings in our taxonomy. Furthermore, we define and discuss these solution concepts under both ESR and SER optimisation criteria. We conclude with a summary of our main findings and a discussion of many promising future research directions in multi-objective multi-agent systems.

Subfields: Computational Theory and Mathematics, Economics and Econometrics, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

1. Patrick Mannion
2. Diederik Marijn Roijers
3. Marcello Restelli
4. Peter Vamplew
5. Richard Dazeley
6. Luisa M. Zintgraf
7. Enda Howley
8. Ann Nowé

In Economics and Econometrics

1. Diederik Marijn Roijers
2. Peter Vamplew
3. Richard Dazeley

In Management Science and Operations Research

1. Marcello Restelli
2. Diederik Marijn Roijers
3. Peter Vamplew
4. Richard Dazeley
5. Patrick Mannion
6. Enda Howley

Citations in the corpus

• Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, neutral): “recent surveys [58], [59].”
• A practical guide to multi-objective reinforcement learning and planning (2021, background): “We note that more complex models exist, such as a multi-objective partially observable Markov decision process [110, 160, 161, 202] and multi-objective multi-agent systems [126].”