Modeling difference rewards for multiagent learning
By Scott Proper, Kagan Tumer. 2012
36 citations recorded by the source corpus.
Difference rewards (a particular instance of reward shaping) have been used to allow multiagent domains to scale to large numbers of agents, but they remain difficult to compute in many domains. We present an approach to modeling the global reward using function approximation that allows the quick computation of shaped difference rewards. We demonstrate how this model can result in significant improvements in behavior for two air traffic control problems. We show how the model of the global reward may be either learned on- or off-line using a linear combination of neural networks.
Subfields: Reinforcement Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Reinforcement Learning
- Karl Tuyls
- Shimon Whiteson
- Joel Z. Leibo
- Thore Graepel
- Marc Lanctot
- Wojciech M. Czarnecki
- Sergey Levine
- Vinícius Flores Zambaldi
Citations in the corpus
- Counterfactual Multi-Agent Policy Gradients (2017, neutral): “r is already being used for learning, difference rewards increase the number of simulations that must be conducted, since each agent’s difference reward requires a separate counterfactual simulation. Proper and Tumer (2012) and Colby, Curran, and Tumer (2015) propose estimating difference rewards us”
- Value-Decomposition Networks For Cooperative Multi-Agent Learning (2017, positive): “This reward has nice properties (e.g. high learnability), but can be impractical as it requires knowledge about the system state (Colby et al., 2016; Agogino and Tumer, 2008; Proper and Tumer, 2012).”
- Local Approximation of Difference Evaluation Functions (2016, unknown): “Difference evaluation functions have been approximated in air traffic control domains [20].”