Evolutionary Function Approximation for Reinforcement Learning

By Shimon Whiteson, Peter Stone. 2006

310 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, methodology): “In practical applications, however, current artificial networks of spiking neurons cannot yet compete with the best traditional deep NNs (e.g., compare MNIST results of Section 5.19).”
A survey and critique of multiagent deep reinforcement learning (2018, methodology): “not realizable in the underlying policy class"[80]. 6Before DQN, many approaches used neural networks for representing the Q-value function [88], such as Neural Fitted Q-learning [87] and NEAT+Q [75]. 7Double Q-learning [91] originally proposed keeping two Qfunctions (estimators) to reduce the ”
Evolutionary Dynamics of Multi-Agent Learning: A Survey (2015, methodology): “Examples of Q-learning algorithms for continuous action spaces are fitted Q-iteration (Ernst, Geurts, & Wehenkel, 2005) and NEAT+Q-learning (Whiteson & Stone, 2006).”
Neuroevolutionary reinforcement learning for generalized control of simulated helicopters (2011, methodology): “Even if methods for improving the on-line performance of neuroevolution [14, 89, 90] were used, such an approach would not be practical.”
Revising the evolutionary computation abstraction: minimal criteria novelty search (2010, methodology): “The NEAT method is appropriate because it is widely applied [1, 3, 31, 32, 34, 35, 37] and well understood.”
Transfer via inter-task mappings in policy search reinforcement learning (2007, methodology): “Empirical results in robot soccer Keepaway and Server Job Scheduling show that TVITM-PS can markedly reduce learning time when full inter-task mappings are available.”
On the deleterious effects of a priori objectives on evolution and representation (2011, background): “This insight is sobering because the vast majority of empirical experiments in EC assess their results based on benchmarks with a priori objectives [2, 8, 13, 20, 24, 27] and theory often focuses on whether targets can be reached [3, 17].”
Adaptive Tile Coding for Value Function Approximation (2007, methodology): “Adaptive methods already exist for neural network function approximators (Whiteson & Stone 2006) and piecewise-linear representations based on kd-trees (Munos & Moore 2002).”
Evolutionary Computation for Reinforcement Learning (2012, methodology): “All three of these implementations of on-line evolutionary computation have been shown to substantially improve on-line performance, both in conjunction with NEAT (Whiteson and Stone, 2006b; Cardamone et al, 2009, 2010) and NEAT+Q (Whiteson and Stone, 2006a).”
A survey of autonomic computing—degrees, models, and applications (2008, methodology): “To this end, a number of hybrid models have been proposed which either speed up training or introduce domain knowledge to reduce the state space, for example, [Tesauro et al. 2006; Whiteson and Stone 2006].”
Abandoning Objectives: Evolution Through the Search for Novelty Alone (2011, methodology): “The NEAT method serves this purpose well because it is both widely applied (Aaltonen et al., 2009; Allen and Faloutsos, 2009; Stanley et al., 2005; Stanley and Miikkulainen, 2002, 2004; Whiteson and Stone, 2006) and well understood.”
Evolution-Guided Policy Gradient in Reinforcement Learning (2018, methodology): “Whiteson and Stone used NEAT [52], an evolutionary algorithm that evolves both neural topology and weights to optimize function approximators representing the value function in Q-learning [60].”
Quantum Reinforcement Learning (2008, methodology): “Many speciﬁc improvements are also implemented to modify related RL methods in practice [7], [9], [10], [20], [21], [22].”
Evolving plastic neural networks with novelty search (2010, background): “…was originally developed to evolve ANNs to solve difficult control and sequential decision tasks and has proven successful in a wide diversity of domains (Aaltonen et al., 2009; Stanley et al., 2003, 2005; Stanley & Miikkulainen, 2002; Taylor, Whiteson, & Stone, 2006; Whiteson & Stone, 2006).”