TDLeaf(lambda): Combining Temporal Difference Learning with Game-Tree Search
By Jonathan Baxter, Andrew Tridgell, Lex Weaver. 1999
In this paper we present TDLeaf(lambda), a variation on the TD(lambda) algorithm that enables it to be used in conjunction with minimax search. We present some experiments in both chess and backgammon which demonstrate its utility and provide comparisons with TD(lambda) and another less radical variant, TD-directed(lambda). In particular, our chess program, ``KnightCap,'' used TDLeaf(lambda) to learn its evaluation function while playing on the Free Internet Chess Server (FICS, fics.onenet.net). It improved from a 1650 rating to a 2100 rating in just 308 games. We discuss some of the reasons for this success and the relationship between our results and Tesauro's results in backgammon.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
Citations in the corpus
- Giraffe: Using Deep Reinforcement Learning to Play Chess (2015, methodology): “The update rule for TD-Leaf( λ ) is as follows - where w is the set of weights of the model, α is the learning rate (set to 1 in our case), ∇ J ( x t , w ) is the gradient of the model at each time point, and d t is the temporal differences from each time point d t to the next time point [21].”