Exploiting structure and utilizing agent-centric rewards to promote coordination in large multiagent systems

By Chris HolmesParker, Adrian K. Agogino, Kagan Tumer. 2013

8 citations recorded by the source corpus.

A goal within the field of multiagent systems is to achieve scaling to large systems involving hundreds or thousands of agents. In such systems the communication requirements for agents as well as the individual agents' ability to make decisions both play critical roles in performance. We take an incremental step towards improving scalability in such systems by introducing a novel algorithm that conglomerates three well-known existing techniques to address both agent communication requirements as well as decision making within large multiagent systems. In particular, we couple a Factored-Action Factored Markov Decision Process (FA-FMDP) framework which exploits problem structure and establishes localized rewards for agents (reducing communication requirements) with reinforcement learning using agent-centric difference rewards which addresses agent decision making and promotes coordination by addressing the structural credit assignment problem. We demonstrate our algorithms performance compared to two other popular reward techniques (global, local) with up to 10,000 agents.

Subfields: Artificial Intelligence, Computer Networks and Communications

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Weinan Zhang
2. Jun Wang
3. Yung Yi
4. Kyunghwan Son
5. Yaodong Yang
6. M. Zhou
7. Daewoo Kim
8. David Earl Hostallero

In Computer Networks and Communications

1. Weinan Zhang
2. Daewoo Kim
3. M. Zhou
4. Yung Yi
5. Kyunghwan Son
6. David Earl Hostallero
7. Davood Hajinezhad
8. Afshin Oroojlooyjadid

Citations in the corpus

• QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning (2019, background): “Multi-domain Gaussian Squeeze (MGS) Gaussian Squeeze (GS) (HolmesParker et al., 2014) is a simple nonmonotonic multi-agent resource allocation problem, used in other papers, e.”
• Mean Field Multi-Agent Reinforcement Learning (2018, background): “In the Gaussian Squeeze (GS) task (HolmesParker et al., 2014), # homogeneous agents determine their individual action 0 9 to jointly optimize the most appropriate summation G = ∑# 9=1 0 9 .”
• A review of cooperative multi-agent deep reinforcement learning (2019, methodology): “In the experimental analysis, the comparisons of QTRAN with VDN and QMIX onMulti-domain Gaussian Squeeze [47] and modified predator-prey [127] is provided.”
• Combining reward shaping and hierarchies for scaling to large multiagent systems (2016, unknown): “…which has been shown to be robust to scaling in a number of domains including air traf fi c control, rover navigation, satellite coordination, and distributed sensor networks (Tumer, 2005; Agogino & Tumer, 2008; Knudson & Tumer, 2010; Agogino et al ., 2012; HolmesParker et al ., 2012, 2013).”