A Theoretical and Empirical Analysis of Reward Transformations in Multi-Objective Stochastic Games

By Patrick Mannion, Jim Duggan, Enda Howley. 2017

11 citations recorded by the source corpus.

Reward shaping has been proposed as a means to address the credit assignment problem in Multi-Agent Systems (MAS). Two popular shaping methods are Potential-Based Reward Shaping and difference rewards, and both have been shown to improve learning speed and the quality of joint policies learned by agents in single-objective MAS. In this work we discuss the theoretical implications of applying these approaches to multi-objective MAS, and evaluate their efficacy using a new multi-objective benchmark domain where the true set of Pareto optimal system utilities is known.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Management Science and Operations Research

Citations in the corpus

Deep Reinforcement Learning for Autonomous Driving: A Survey (2020, background): “Both D and PBRS have been successfully applied to a wide range of application domains and have the added beneﬁt of convenient theoretical guarantees, meaning that they do not suffer from the same issues as the unprincipled reward shaping approaches described above (see e.g. [49]–[53]).”
Multi-objective multi-agent decision making: a utility-based analysis and survey (2019, background): “t al. [60] provide a theoretical analysis of PBRS in singleand multi-agent MORL settings, demonstrating that the Pareto relation between (joint) policies is preserved when PBRSis used. Mannion et al. [62] also provide 9 Or, in their original paper, the equivalent concept of domination. 10 Although i”
Policy invariance under reward transformations for multi-objective reinforcement learning (2017, background): “Recent work has made some progress towards achieving this goal [34, 35], and this line of research should be continued in order to produce a comprehensive suite of multi-agent MORL benchmark domains.”
Ordered Preference Elicitation Strategies for Supporting Multi-Objective Decision Making (2018, background): “We hope that this work inspires new real-world applications involving multiple objectives, and will be used in multi-objective optimisation [15, 16, 23, 29], planning [4, 41, 48, 53], and RL [2, 33, 42, 52] alike.”