Efficient memory-based learning for robot control

By Andrew W. Moore. 1990

Subfields: Reinforcement Learning, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

In Robotics & Control

Citations in the corpus

Benchmarking Deep Reinforcement Learning for Continuous Control (2016, methodology): “…(Stephenson, 1908; Donaldson, 1960; Widrow, 1964; Michie & Chambers, 1968), Cart-Pole Swing Up (Kimura & Kobayashi, 1999; Doya, 2000), Mountain Car (Moore, 1990), Acrobot Swing Up (DeJong & Spong, 1994; Murray & Hauser, 1991; Doya, 2000), and Double Inverted Pendulum Balancing (Furuta et al.,…”
Generative Adversarial Imitation Learning (2016, background): “…on 9 physics-based control tasks, ranging from low-dimensional control tasks from the classic RL literature—the cartpole [2], acrobot [8], and mountain car [17]—to difﬁcult high-dimensional tasks such as a 3D humanoid locomotion, solved only recently by model-free reinforcement learning [27, 26].”
Designing Interpretable Approximations to Deep Reinforcement Learning with Soft Decision Trees (2020, methodology): “The mountain car problem is a well-known benchmark problem in reinforcement learning, prevalent in the literature since the 1990s (Moore 1990).”
A General Family of Robust Stochastic Operators for Reinforcement Learning (2018, neutral): “This problem is first discussed in [14].”
Behaviour Suite for Reinforcement Learning (2019, background): “‘CartPole’ (Barto et al., 1983) and ‘MountainCar’ (Moore, 1990) are examples of classic benchmarks that has provided a testing ground for RL algorithm development.”
A Brief Survey of Deep Reinforcement Learning (2017, neutral): “much early work focused on simple, custom MDPs, there shortly emerged control problems that could be used as standard benchmarks for testing new algorithms, such as the Cartpole [5], Acrobot [22] and Mountain Car [74] domains.”
Interpreting Blackbox Models via Model Extraction (2017, methodology): “For the cartpole (discrete actions) [2, 10], the learner uses a discretized state space (7 bins per dimension) [1], and for the mountain car (discrete actions) [4, 33], it approximates the Q-function with a linear model over RBF features [3].”
DORA The Explorer: Directed Outreaching Reinforcement Action-Selection (2018, methodology): “To test whether E -values can serve as generalized visit-counters in the function-approximation case, we used a linear approximation architecture on the MountainCar problem (Moore, 1990) (Appendix Published as a conference paper at ICLR 2018 C).”
An approach to reachability analysis for feed-forward ReLU neural networks (2017, background): “These are well-known problems from classic control theory [29, 6, 23].”
Learning to Act Using Real-Time Dynamic Programming (1995, methodology): “Similarly, using the kd-tree data structure to access state costs, as explored by Moore [55,56], can provide efficient storage and retrieval of the costs of a finite set of states from a k-dimensional state space.”
Planning by Incremental Dynamic Programming (1991, background): “Dyna-like systems that go beyond the tabular case have been investigated by Lin (1991), Moore (1990), Riolo (1991), and Watkins (1989).”
The three-dimensional normal-distributions transform : an efficient representation for registration, surface analysis, and loop detection (2009, background): “Use of kD trees does not scale very well to higher dimensions, and in spaces with more than eight or so dimensions, using kD trees does not generally improve performance over brute-force search [74].”
Flexible Option Learning (2021, background): “The environment we consider is a sparse variant of the classic Mountain Car domain [Moore, 1991, Sutton and Barto, 2018] where a positive reward is given upon reaching the goal.”
A Survey of Imitation Learning Methods, Environments and Metrics (2024, background): “Alongside the CartPole, the most common environments for discrete actions are the MountainCar [71] environment and Atari games [12].”
Learning to Learn (1998, unknown): “A popular example of learning control parameters can be found in the context of memory-based methods such as nearest neighbor [Franke, 1982; Moore, 1990; Stanfill and Waltz, 1986].”
Improving Generalization for Temporal Difference Learning: The Successor Representation (1993, methodology): “Both Moore (1990) and Chapman and Kaelbling (1991) sought to ex-orcise the daemon of dimensionality by using better function approximation schemes, which is an equivalent step to using a simple linear scheme with more sophisticated input representations.”
Assessing Generalization in Deep Reinforcement Learning (2018, background): “Three environment parameters can be varied: (1) push force magnitude, (2) pole length, and (3) pole mass. MountainCar (Moore, 1990).”
Scalable Techniques from Nonparametric Statistics for Real Time Robot Learning (2002, background): “…most of the p training data points receive an approximately zero weight as they are too far away from the query point, the computational complexity of LWR can be reduced significantly, particularly when exploiting efficient data structures like kd-trees for keeping the data in memory (Moore, 1990).”