Learning Tetris Using the Noisy Cross-Entropy Method

By Istvan Szita, András Lörincz. 2006

259 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Reinforcement Learning

Citations in the corpus

Proximal Policy Optimization Algorithms (2017, methodology): “We compared against tuned implementations of the following algorithms: trust region policy optimization [Sch+15b], cross-entropy method (CEM) [SL06], vanilla policy gradient with adaptive stepsize 3 , 2 HalfCheetah, Hopper, InvertedDoublePendulum, InvertedPendulum, Reacher, Swimmer, and Walker2d,…”
Trust Region Policy Optimization (2015, methodology): “…have a close connection to policy iteration); and (3) derivative-free optimization meth-ods, such as the cross-entropy method (CEM) and covariance matrix adaptation (CMA), which treat the return as a black box function to be optimized in terms of the policy parameters (Szita & L¨orincz, 2006).”
Benchmarking Deep Reinforcement Learning for Continuous Control (2016, methodology): “We implement a range of gradient-based policy search meth-ods, as well as two gradient-free methods for comparison with the gradient-based approaches.”
A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation (2016, neutral): “The policy search can be carried out in two paradigms: gradient-free (Fu et al., 2005; Szita and L¨orincz, 2006) and gradient-based.”
Episodic Exploration for Deep Deterministic Policies: An Application to StarCraft Micromanagement Tasks (2016, methodology): “Compared to algorithms for eﬃcient direct exploration in parameter space (see e.g., [18, 27, 37, 25]), the novelty of our algorithm is to mix exploration through parameter randomization and plain gradient descent.”
Approximate modified policy iteration and its application to the game of Tetris (2015, methodology): “Figure 7 contains the pseudo-code of the CE algorithm used in our experiments (Szita and Lőrincz, 2006; Thiery and Scherrer, 2009b).”
Approximate Dynamic Programming Finally Performs Well in the Game of Tetris (2013, unknown): “CE is the state-of-the-art method in Tetris with huge performance advantage over ADP/RL methods [18, 19, 20].”
Reinforcement Learning in Continuous State and Action Spaces (2012, background): “Some versions of cross-entropy add noise to the variance to prevent premature convergence (e.g., Szita and Lörincz, 2006), but the theory behind this seems less well-developed than covariance estimation used by CMA-ES.”
The Reinforcement Learning Competitions (2010, background): “Tetris The Tetris domain (Bertsekas and Tsitsiklis 1996; Demaine, Hohenberger, and Liben-Nowell 2003; Szita and Lörincz 2006), depicted in figure 3, is based on the popular falling-blocks puzzle game.”
Policy Optimization by Genetic Distillation (2017, methodology): “Cross-Entropy methods use similar ideas and have been found to work reasonably well in simple environments (Szita & L¨orincz, 2006).”
Contrasting Exploration in Parameter and Action Space: A Zeroth-Order Optimization Perspective (2019, result): “This matches the intuition and results presented in recent works like (Bagnell and Schneider, 2001; Szita and Lörincz, 2006; Tesch et al., 2011; Salimans et al., 2017; Mania et al., 2018) that show parameter space black-box policy search methods outperforming state-of-the-art action space methods fo”
Interactively shaping agents via human reinforcement: the TAMER framework (2009, background): “Policy search algorithms, which do not model a value or reinforcement function, attain the best final performance, reaching hundreds of thousands of lines per game [5, 17], though they require many games to get there and do not learn within the first 500 games.”
TAMER: Training an Agent Manually via Evaluative Reinforcement (2008, methodology): “A linear combination of these features has been used successfully by a number of other researchers since and described in papers including [16] (results shown in Table I).”