Federated Control with Hierarchical Multi-Agent Deep Reinforcement Learning

By Saurabh Kumar, Pararth Shah, Dilek Z. Hakkani-Tür, Larry Heck. 2017

36 citations recorded by the source corpus.

We present a framework combining hierarchical and multi-agent deep reinforcement learning approaches to solve coordination problems among a multitude of agents using a semi-decentralized model. The framework extends the multi-agent learning setup by introducing a meta-controller that guides the communication between agent pairs, enabling agents to focus on communicating with only one other agent at any step. This hierarchical decomposition of the task allows for efficient exploration to learn policies that identify globally optimal solutions even as the number of collaborating agents increases. We show promising initial experimental results on a simulated distributed scheduling problem.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Multi-Agent Common Knowledge Reinforcement Learning (2018, background): “Liu et al. (2017) investigate how to learn such models in environments where the transition dynamics are not known. Makar, Mahadevan, and Ghavamzadeh (2001) extend the MAXQ single-agent hierarchical framework by Dietterich (2000) to the multi-agent domain.”
Multi-agent deep reinforcement learning: a survey (2021, background): “A deep approach was proposed by Kumar et al. (2017), where a higher-level controller guides the information exchange between decentralized agents.”
Feudal Multi-Agent Hierarchies for Cooperative Reinforcement Learning (2019, methodology): “A meta-controller which structures communication between agent pairs in order to achieve coordination has also been proposed [14] (and could naturally be hybridized with FMH), as well as the use of master-slave architecture which merges the actions of a centralised master agent with those of decentr”
Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications (2018, background): “was developed to coordinate and control multiple agents in problems where agents’ privacy is prioritized [92].”
Hierarchical Deep Multiagent Reinforcement Learning (2018, background): “Deep Reinforcement Learning (DRL) has achieved a great number of successes in recent years (Mnih et al. 2015; Levine et al. 2016; Gu et al. 2017).”
When Does Communication Learning Need Hierarchical Multi-Agent Deep Reinforcement Learning (2019, background): “Although deep reinforcement learning algorithms are successful in many single-agent scenarios, as the problems become more complex and multiple agents in a single environment must interact, the algorithms do not scale well (Kumar et al. 2017).”