Local Approximation of Difference Evaluation Functions

By Mitchell K. Colby, Theodore Duchow-Pressley, Jen Jen Chung, Kagan Tumer. 2016

22 citations recorded by the source corpus.

Difference evaluation functions have resulted in excellent multiagent behavior in many domains, including air traffic and mobile robot control. However, calculating difference evaluation functions requires determining the value of a counterfactual system objective function, which is often difficult when the system objective function is unknown or global state and action information is unavailable. In this work, we demonstrate that a local estimate of the system evaluation function may be used to estimate difference evaluations using readily available information, allowing for difference evaluations to be computed in multiagent systems where the mathematical form of the objective function is not known. This approximation technique is tested in two domains, and we demonstrate that approximating difference evaluation functions results in better performance and faster learning than when using global evaluation functions. Finally, we demonstrate the effectiveness of the learned policies on a set of Pioneer P3-DX robots.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Mathematical Optimization

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Sergey Levine
2. Wojciech M. Czarnecki
3. Karl Tuyls
4. Thore Graepel
5. Joel Z. Leibo
6. Marc Lanctot
7. Max Jaderberg
8. Shimon Whiteson

In Computational Theory and Mathematics

1. Wojciech M. Czarnecki
2. Thore Graepel
3. Joel Z. Leibo
4. Peter Sunehag
5. Karl Tuyls
6. Max Jaderberg
7. Guy Lever

In Mathematical Optimization

1. Wojciech M. Czarnecki
2. Thore Graepel
3. Max Jaderberg

Citations in the corpus

• Value-Decomposition Networks For Cooperative Multi-Agent Learning (2017, background): “This reward has nice properties (e.g. high learnability), but can be impractical as it requires knowledge about the system state (Colby et al., 2016; Agogino and Tumer, 2008; Proper and Tumer, 2012).”
• Value-Decomposition Networks For Cooperative Multi-Agent Learning Based On Team Reward (2018, background): “Difference rewards [24] measures the impact of an agent’s action on the team reward, but comes with practical difficulties [1, 6, 18].”