Theoretical Advantages of Lenient Learners: An Evolutionary Game Theoretic Perspective

By Liviu Panait, Karl Tuyls, Sean Luke. 2008

98 citations recorded by the source corpus.

Subfields: Management Science and Operations Research, Safety Research, Sociology and Political Science, Game & Decision Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Management Science and Operations Research

In Safety Research

In Sociology and Political Science

In Game & Decision Theory

Citations in the corpus

Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2020, background): “Panait et al. (2008) introduce Leniency which ignores updates that decrease Q -value estimates with a probability that is decreasing during training.”
QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning (2018, background): “ estimates, which is slightly more robust to stochasticity and the presence of multiple optimal joint actions. Omidshaei et al. (2017) utilises Hysteretic Q-learning in a multi-task deep RL setting. Panait et al. (2008) introduce Leniency which ignores updates that decrease Q-value estimates with a ”
Action Semantics Network: Considering the Effects of Actions in Multiagent Systems (2019, background): “More recently, Palmer et al. (2018) extended the idea of leniency (Potter & Jong, 1994; Panait et al., 2008) to deep MARL and proposed the retroactive temperature decay schedule to address stochastic rewards problems.”
Lenient Multi-Agent Deep Reinforcement Learning (2017, methodology): “Lenient learners do so by effectively forgiving (ignoring) sub-optimal actions by teammates that lead to low rewards during the initial exploration phase [22, 23].”
Multiagent Learning: Basics, Challenges, and Prospects (2012, background): “A prime example is the formal link between coevolutionary algorithms and multiagent Q-learning established through the replicator equations from evolutionary game theory (EGT) (Panait, Tuyls, and Luke 2008), but also relations between swarm intelligence and reinforcement learning are being discovere”
Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problems (2012, methodology): “The formal deﬁnition we propose here is closer to the concept described by Panait, Tuyls, and Luke (2008).”
A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning (2017, background): “use adaptive learning rates [12], adjust the frequencies of updates [48, 81], or dynamically respond to the other agents actions online [63, 50].”
A survey and critique of multiagent deep reinforcement learning (2018, background): “ient-DQN (LDQN) [35] took the leniency concept [222] (originally presented in MAL) and extended their use to MDRL.The purpose of leniency is to overcome a pathology called relative overgeneralization [34, 223, 224]. Similar to other approaches designed to overcome relative overgeneralization (e.g., ”
Evolutionary Dynamics of Multi-Agent Learning: A Survey (2015, background): “This can be achieved by collecting κ rewards for each action before performing an update based on the highest of those rewards (Panait et al., 2008; Bloembergen et al., 2011).”
Lenient Learning in Independent-Learner Stochastic Cooperative Games (2016, methodology): “Relative overgeneralization is a specific and problematic subcase of the slightly more general notion of action shadowing, occasionally used by the MARL community, but has only been relatively recently studied (Panait et al., 2006b; Panait, 2006; Fulda and Ventura, 2007; Panait et al., 2008).”
Dynamic analysis of multiagent Q-learning with ε-greedy exploration (2009, background): “The same type of link has also been explored by Borgers and Sarin (1997) and Panait et al. (2008) .”
An Evolutionary Game Theoretic Analysis of Difference Evaluation Functions (2015, methodology): “Thus, assessing the fitness of each agent is context-sensitive and subjective [8].”
Prosocial learning agents solve generalized Stag Hunts better than selfish ones (2017, result): “Our results relate to other solutions to the multiple optima problem discussed in past work on multi-agent systems in particular on lenient learning (Panait, Tuyls, and Luke 2008; Matignon, Laurent, and Le Fort-Piat 2012) or Frequency Maximum Q-learning (Kapetanakis and Kudenko 2002).”
Empirical and theoretical support for lenient learning (2011, background): “, forgiving initial mis coordination, can greatly improve the accuracy of an agent’ s reward estimation in the beginning of the learning process [4].”
Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence (2019, background): “Many ideas have been raised to address such issues [10, 42, 43, 55, 72].”
Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications (2018, background): “In this situation, with the leniency, agent A will think that agent B can improve his skills and thus agent A continues passing the ball to agent B instead of thinking that agent B does not have skills to play football and not passing the ball to agent B again [56].”
The world of independent learners is not markovian (2011, methodology): “We can mention the WOLF-PHC algorithm [4], the Hysteretic Q-Learning [17], and lenient learners [25].”
Learning the IPA market with individual and social rewards (2007, unknown): “Evolutionary Game Theory [19,26] may be an alley for additional theoretical exploration.”