Experiments with Reinforcement Learning in Problems with Continuous State and Action Spaces

By Juan Carlos Santamaría, Richard S. Sutton, Ashwin Ram. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Deep learning in neural networks: An overview (2014, methodology): “Related methods include probability distributionbased EAs (Baluja, 1994; Larraanaga & Lozano, 2001; Sałustowicz & Schmidhuber, 1997; Saravanan & Fogel, 1995), Covariance Matrix Estimation Evolution Strategies (CMA-ES) (Hansen, Müller, & Koumoutsakos, 2003; Hansen & Ostermeier, 2001; HeidrichMeisner ”
On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models (2015, methodology): “The parameters or weights of C are trained to maximize reward by a standard RL method such as Q-learning or similar methods [17, 292, 293, 172, 254, 212, 262, 10, 122, 188, 157, 281, 26, 216, 197, 272, 311, 9, 163, 174, 22, 27, 2, 137, 276, 156, 284].”
Reinforcement Learning with Factored States and Actions (2004, background): “The state or action space could also be continuous (Baird and Klopf, 1993; Santamaria et al., 1998).”
Robust non-linear control through neuroevolution (2003, methodology): “Sarsa(λ) with Case-Based function approximator (SARSA-CABA; Santamaria et al. 1998): This method consists of the Sarsa on-policy Temporal Difference control algorithm with eligibility traces that uses a case-based memory to approximate the Q-function.”
Real-time neuroevolution in the NERO video game (2005, methodology): “Traditional reinforcement learning (RL) techniques such as Q-Learning [28] and Sarsa with a case-based function approximator (SARSA-CABA; [29]) adapt in domains with sparse feedback [28], [30], [31].”
Improving Action Selection in MDP's via Knowledge Transfer (2005, background): “However, large state and action sets remain a stumbling block for RL. While large state sets have seen much work in recent research (Tesauro 1994; Crites & Barto 1996; Stone & Sutton 2001), large action sets have been explored to but a limited extent ( Santamaria, Sutton, & Ram 1997; Gaskett, Wette”
Accelerated Neural Evolution through Cooperatively Coevolved Synapses (2008, methodology): “Sarsa(λ) with Case-Based function approximator (SARSA-CABA; Santamaria et al., 1998): This method consists of the Sarsa on-policy Temporal Difference control algorithm with eligibility traces that uses a case-based memory to approximate the Q-function.”
Learning from observation using primitives (2001, methodology): “[111], Smart and Kaelbling [118], and Forbes and Andre [44] give some insight on various methods that can be used to encode the Q value in a function approximator.”
A comparison of direct and model-based reinforcement learning (1997, background): “We learn online, without a model, and back up whatever states are encountered during 40 second long trials (200 steps) with the following equations (Sutton, 1996; Santamaria et al., 1996):”
Neuroevolution strategies for episodic reinforcement learning (2009, methodology): “If after 10 evaluations no successful control strategy was found, a trial was regarded as a failure In such a case, the number of evaluations was set to 10 when computing the”
Reinforcement Learning in Continuous State and Action Spaces (2012, methodology): “An example of such a discretizing method that is often used inrei forcement learning istile coding (Watkins, 1989; Lin and Kim, 1991; Sutton, 1996; Santamaria et al, 1997; Sutton and Barto, 1998), which is based on the Cerebellar Model Articulation Controller (CMAC) structure proposed by Albus…”
Learning From Observation and Practice Using Behavioral Primitives : Marble Maze (2004, background): “[9, 12, 14, 16] are examples of work in reinforcement learning that use memory-based approaches to represent the value function itself.”