Multi-Agent Reinforcement Learning:a critical survey

By Yoav Shoham, Rob Powers, Trond Grenager. 2003

324 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/a69e33ce29c451655d917bf9387ed57e115fcfc7

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Resource allocation in the grid using reinforcement learning (2004, background): “In the multi-agent setting, however, the environment is highly dynamic because of the presence of other learning agents, and the usual conditions for convergence to an optimal policy do not necessarily hold [6, 17, 24 ].”
A Comprehensive Survey of Multiagent Reinforcement Learning (2008, background): “However, the suitability of such stage-wise solutions in the context of t he dynamic task is currently unclear [14], [17].”
QD-Learning: A Collaborative Distributed Strategy for Multi-Agent Reinforcement Learning Through Consensus + Innovations (2012, background): “There has been extensive research on multi-agent reinforcement learning (see [9], [10] for surveys).”
Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations (2020, methodology): “[17] applies MARL [19] to train the routing network.”
Deep Reinforcement Learning: An Overview (2017, background): “…al., 2012; He and Deng, 2013; Young et al., 2013); AI safety (Amodei et al., 2016; Garc`ıa and Fern`andez, 2015), Monte Carlo tree search (MCTS) (Browne et al., 2012; Gelly et al., 2012); multi-agent RL (Shoham et al., 2003; Busoniu et al., 2008); game theory (Leyton-Brown and Shoham, 2008), etc.”
Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspective (2019, methodology): “,a). Similarly, the reward function R(s) is encoded in a probability distribution 6 P(R t = r | S t = s). Commonly, in RL, the transition and reward functions are initially unknown to the agent (e.g. Shoham et al., 2004). This can be modeled by letting the transition and reward functions depend on u”
Inverse Reinforcement Learning in Swarm Systems (2016, background): “The rationale is that, unlike in most single-agent or smallscale problems involving complicated agent architectures, the complexity of such networks typically arises from the sheer amount of local interactions, making the resulting learning problems particularly hard to solve [2, 7, 27].”
An Overview of Cooperative and Competitive Multiagent Learning (2005, background): “opponent agents have to be given for which best learning strategies must be determined [148] (the “AI agenda”), while other robust evaluation criteria for resulting strategies must be determined and satisfied (e.”
Imitation Learning (2017, background): “Although creating autonomous multiagents has been thoroughly investigated through reinforcement learning [Shoham et al. 2003; Busoniu et al. 2008], it is not as extensively explored in imitation learning.”
Nash Q-Learning for General-Sum Stochastic Games (2003, background): “Shoham et al. (2003) take issue with the focus on convergence toward equilibrium, and propose four alternative, well-defined problems that multiagent learning might plausibly address.”
Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents (2018, background): “This improves the scalability of the multi-agent model with a high population of agents, which is one of the long-standing challenges in general MARL problems (Shoham et al., 2003).”
Convergence and No-Regret in Multiagent Learning (2004, background): “Convergence as an important criterion has recently been called into question (e.g., ( Shoham et al., 2003 )).”
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms (2019, background): “In fact, as argued in [122], the unclarity of the problems being addressed is the fundamental flaw in many early MARL works.”
Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial (2020, background): “Nevertheless, as outlined by [7], solving multi-agent scenarios using NE is problematic for several reasons.”
Multiagent Reinforcement Learning for Multi-Robot Systems: A Survey (2004, background): “A great deal of research on multiagent learning has borrowed the theoretic frameworks and notions from SGs [33]–[37], [39], [41]–[45], [47]–[50], [55]– [58].”
Opportunities for multiagent systems and multiagent reinforcement learning in traffic control (2009, background): “Related to this discussion, two issues discussed in [69] are important from the perspective of traffic control.”
CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learning (2018, background): “While the framework of multi-agent reinforcement learning (MARL) [40, 17, 35, 9, 30] has been equipped with successful methods in deep reinforcement learning (RL) [20, 15] and shown promise on high-dimensional problems with complex agent interactions [19, 21, 13, 6, 16, 34], learning multi-agent coo”