Hierarchical multi-agent reinforcement learning

By Mohammad Ghavamzadeh, Sridhar Mahadevan, Rajbala Makar. 2001

282 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Automotive Engineering, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Automotive Engineering

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning (2017, background): “However, many real-world problems, such as network packet delivery (Ye et al., 2015), rubbish removal (Makar et al., 2001), and urban trafﬁc control (Kuyer et al., 2008; Van der Pol & Oliehoek, 2016), are naturally modeled as cooperative multi-agent systems.”
Episodic Exploration for Deep Deterministic Policies: An Application to StarCraft Micromanagement Tasks (2016, background): “…policies for individual agents in a collaborative setting with communication constraints [38, 3], with applications such as soccer robot control [31], and methods such as hierarchical reinforcement learning for communicating high-level goals [12], or learning an eﬃcient communication protocol [32].”
Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discovery (2019, methodology): “methods in early work [12], while their final performance are within the margin of error (Figure 4a).”
The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning (2019, background): “In future applications, intelligent agents will cooperate and/or compete as part of multi-agent systems (MASs) [30, 15, 33, 7].”
Multi-Agent Common Knowledge Reinforcement Learning (2018, neutral): “Makar et al. (2001) extend the MAXQ single-agent hierarchical framework (Di-etterich, 2000) to the multi-agent domain.”
Negative Update Intervals in Deep Multi-Agent Reinforcement Learning (2018, methodology): “Similar to Ghavamzadeh et al. (2006) we manually specify cooperation strategies and incorporate them as domain knowledge.”
Hierarchical Reinforcement Learning: A Survey (2015, methodology): “Various HRL models are available like MAXQ, Hierarchical abstract machines (HAMs), ALisp and options [26], [33]; the models that scale RL to large state spaces problems by decomposing them into subproblems.”
Multi-agent Reinforcement Learning: An Overview (2010, methodology): “Knowledge about the task structure can be used to decompose it in subtasks, and learn a modular solution with e.g., hierarchical RL [40].”
A review of cooperative multi-agent deep reinforcement learning (2019, methodology): “They have analyzed their algorithm in Multiagent Trash Collection tasks (an extension of environment [128]) and Fever Basketball Defense.”
Feudal Multi-Agent Hierarchies for Cooperative Reinforcement Learning (2019, methodology): “The MAXQ algorithm [6] has been used to train homogenous agents [19], allowing them to coordinate by communicating subtasks rather than primitive actions, an idea recently re-explored in the context of deep RL [30].”
Planning with macro-actions in decentralized POMDPs (2014, unknown): “[13].”
Recent Advances in Hierarchical Reinforcement Learning (2003, background): “Among the termination schemes studied by Rohanimanesh and Mahadevan [57] are the following two.”
Learning of coordination: exploiting sparse interactions in multiagent systems (2009, background): “In [6], the authors propose a hierarchical multiagent RL algorithm that takes advantage of subtasks that can be learned individually by the agents, allowing coordination to be addressed only at the level of the main task.”
Learning multi-agent state space representations (2010, unknown): “in [6] coordination among agents is done at the level of primitive actions and not at a subtask level since this requires prior knowledge of the problem at hand.”
Multiagent Learning of Coordination in Loosely Coupled Multiagent Systems (2015, unknown): “For example, hierarchical MAL approaches have been developed to take advantage of agents’ structural dependence to speed up the skill acquisition in cooperative MASs [16]–[18].”
Hierarchical Deep Multiagent Reinforcement Learning (2018, methodology): “We devise a range of two-player grid-world domains, i.e., Multiagent Trash Collection (MATC) tasks, by extending the classic game in (Makar, Mahadevan, and Ghavamzadeh 2001).”