Extending Q-Learning to General Adaptive Multi-Agent Systems

By G. Tesauro. 2003

242 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments (2017, methodology): “Q-Learning and DQN [23] are popular methods in reinforcement learning and have been previously applied to multi-agent settings [8, 35].”
Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning (2017, methodology): “Methods like hyper Q-learning (Tesauro, 2003) - which we also discuss in Section 3.2 - and AWESOME (Conitzer & Sandholm, 2007) try to tackle the nonstationarity by tracking and conditioning each agents’ learning process on their teammates’ current policy, while Da Silva et al. (2006) propose…”
Empirically Evaluating Multiagent Learning Algorithms (2014, methodology): “f algorithms was Powers and Shoham (2005). While four of the eleven algorithms tested in this study were simple stationary-strategy baselines, the remaining seven were MALalgorithms including Hyper-Q(Tesauro, 2004), WoLF-PHC (Bowling and Veloso, 2002), and a joint action learner (Claus and Boutilier”
Factorized Q-learning for large-scale multi-agent systems (2018, background): “h state and •nd the equilibrium. Besides, when there are many agents, the estimated policies of di‡erent agents might not belong to the same Nash equilibrium, thus the convergence will become invalid [25]. In the best-response learning paradigm, each agent just tries to learn a policy that is optima”
A survey and critique of multiagent deep reinforcement learning (2018, background): “Note that instead of convergence, some MAL algorithms have proved learning a best response against classes of opponents [150, 151, 152].”
Evolutionary Dynamics of Multi-Agent Learning: A Survey (2015, background): “Examples of joint action learners are minimax-Q (Littman, 1994), fictitious play and AWESOME (Brown, 1951; Conitzer & Sandholm, 2007), hyper-Q (Tesauro, 2003), and Nash-Q (Hu & Wellman, 2003).”
Influencing Long-Term Behavior in Multiagent Reinforcement Learning (2022, unknown): “However, a model-based approach has difficulties in addressing the infinite recursion problem: if an agent attempts to take into account its opponent’s model of the agent itself (Tesauro, 2004).”
Spectrum Sharing in Vehicular Networks Based on Multi-Agent Reinforcement Learning (2019, background): “This means we can augment each agent’s observation space with an estimate of other agents’ policies to avoid nonstationarity, which is the essential idea of hyper Qlearning [33].”
Deep Multi Agent Reinforcement Learning for Autonomous Driving (2020, background): “This change allows the agents to condition their policy on the changing parameters of the other agents and thus avoid the problem of non-stationary environments in MARL [43].”
Cooperative Multi-agent Control Using Deep Reinforcement Learning (2017, background): “This line of work combines ideas from deep learning with earlier work on function approximation [40, 22], giving rise to the field of deep reinforcement learning.”
An Overview of Cooperative and Competitive Multiagent Learning (2005, background): “An example of the above is that Hyper-Q [170].”
Deep Reinforcement Learning for User Association and Resource Allocation in Heterogeneous Cellular Networks (2019, unknown): “Bellman optimality equation [41]:”
Robust Multi-Agent Reinforcement Learning via Minimax Deep Deterministic Policy Gradient (2019, methodology): “2015) are popular methods in reinforcement learning and have been previously applied to multi-agent settings (Foerster et al. 2016a; Tesauro 2004).”
Two Body Problem: Collaborative Visual Task Completion (2019, background): “Multiple approaches have been proposed over the years to address such concerns [82, 83, 81, 30].”
A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks (2020, methodology): “Multiple methods have been proposed to address resulting issues [88,89,87,29].”
Joint Policy Search for Multi-agent Collaboration with Imperfect Information (2020, unknown): “Previous works also model belief space (e.g., Point-Based Value Iteration [31] in POMDP, BAD [16]) or model the behaviors of other agents (e.g., AWESOME [11], Hyper Q-learning [37], LOLA [15]).”