Giraffe: Using Deep Reinforcement Learning to Play Chess

By Matthew Lai. 2015

115 citations recorded by the source corpus.

This report presents Giraffe, a chess engine that uses self-play to discover all its domain-specific knowledge, with minimal hand-crafted knowledge given by the programmer. Unlike previous attempts using machine learning only to perform parameter-tuning on hand-crafted evaluation functions, Giraffe's learning system also performs automatic feature extraction and pattern recognition. The trained evaluation function performs comparably to the evaluation functions of state-of-the-art chess engines - all of which containing thousands of lines of carefully hand-crafted pattern recognizers, tuned over many years by both computer chess experts and human chess masters. Giraffe is the most successful attempt thus far at using end-to-end machine learning to play chess.

Subfields: Artificial Intelligence, Economics and Econometrics, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Economics and Econometrics

In Reinforcement Learning

Citations in the corpus

VAIN: Attentional Multi-agent Predictive Modeling (2017, neutral): “Approaches include [24, 25] for Chess, [26, 27, 28] for Backgammons [29] for Go.”
Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm (2017, background): “Giraffe (20) evaluated positions by a neural network that included mobility maps and attack and defend maps describing the lowest valued attacker and defender of each square.”
Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning (2017, background): “Deep learning has also been used as a component in more complex systems that are able to play games [32, 40, 55, 58] or diagnose and classify diseases [15, 17, 25].”
Interactive reinforcement learning for task-oriented dialogue management (2016, unknown): “Go (Silver et al. (2016)) and chess (Lai (2015)).”
Flow: Architecture and Benchmarking for Reinforcement Learning in Traffic Control (2017, background): “Deep reinforcement learning (RL), which is the main enabler in our framework, is a powerful tool for control and has already had demonstrated success in complex but data-rich problem settings such as Atari games [28], 3D locomotion and manipulation [29], [30], [31], chess [32], among others.”
Neural Networks and Deep Learning (2018, unknown): “Some TD-learning methods for chess, such as NeuroChess [496], KnightCap [22], and Giraffe [259] have been explored, but were not as successful as conventional engines.”
PassGAN: A Deep Learning Approach for Password Guessing (2017, background): “Deep learning has been extensively used in various areas of computer science, including computer vision [51], image processing [85], video processing [21], [65], speech recognition [34], natural language processing [2], [15], [95], and gaming [32], [48], [59], [62].”
Mastering the game of Go without human knowledge (2017, background): “Selfplay reinforcement learning approaches have achieved high levels of perfor mance in other games: chess 49–51 , checkers 52 , backgammon 53 , othello 54 , Scrabble 55 and most recently poker 56 .”
DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess (2016, unknown): “Although reinforcement learning has also been applied to chess [1,10], the resulting programs exhibit a playing strength at a human master level at best, which is substantially lower than the grandmaster-level state-of-the-art chess programs.”