Empirical and theoretical support for lenient learning

By Daan Bloembergen, Michael Kaisers, Karl Tuyls. 2011

12 citations recorded by the source corpus.

Recently, an evolutionary model of Lenient Q-learning (LQ) has been proposed, providing theoretical guarantees of convergence to the global optimum in cooperative multi-agent learning. However, experiments reveal discrepancies between the predicted dynamics of the evolutionary model and the actual learning behavior of the Lenient Q-learning algorithm, which undermines its theoretical foundation. Moreover it turns out that the predicted behavior of the model is more desirable than the observed behavior of the algorithm. We propose the variant Lenient Frequency Adjusted Qlearning (LFAQ) which inherits the theoretical guarantees and resolves this issue.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Lenient Multi-Agent Deep Reinforcement Learning (2017, background): “Leniency has been shown to increase the likelihood of convergence towards the globally optimal solution in stateless coordination games for reinforcement learning agents [4, 22, 23].”
Evolutionary Dynamics of Multi-Agent Learning: A Survey (2015, methodology): “Regret minimisation (RM) is omitted as its dynamics are visually indistinguishable from Cross Bloembergen, Tuyls, Hennes, & Kaisers FAQ-learning 0 0.25 0.5 0.75 1 0 0.25 0.5 0.75 1 x 1 y 1 0 0.25 0.5 0.75 1 0 0.25 0.5 0.75 1 x 1 y 1 0 0.25 0.5 0.75 1 0 0.25 0.5 0.75 1 x 1 y 1 Lenient FAQ-learning 0 ”
Weighted Double Deep Multiagent Reinforcement Learning in Stochastic Cooperative Environments (2018, background): “Approaches like [Bloembergen et al. , 2011; Matignon et al. , 2007; Matignon et al. , 2012; Panait et al. , 2006; Wei and Luke, 2016 ] have been proposed by extending Q-learning to address the coordination problems in cooperative multiagent systems.”