A comparison of direct and model-based reinforcement learning

By Christopher G. Atkeson, Juan Carlos Santamaría. 1997

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Reinforcement Learning

Citations in the corpus

Model-Ensemble Trust-Region Policy Optimization (2018, background): “Previous work has pointed out this issue as model bias , i.e. (Deisenroth & Rasmussen, 2011; Schneider, 1997; Atkeson & Santamaria, 1997).”
Successor Features for Transfer in Reinforcement Learning (2016, background): “By specializing the basic components presented here one can build on our results to derive agents able to perform well across a wide variety of tasks, and thus handle environments of considerable complexity.”
Graph networks as learnable physics engines for inference and control (2018, methodology): “…planning in a similar spirit to classic approaches which use pre-deﬁned models (Li & Todorov, 2004; Tassa et al., 2008; 2014), and our work also relates to learning-based approaches for model-based control (Atkeson & Santamaria, 1997; Deisenroth & Ras-mussen, 2011; Levine & Abbeel, 2014).”
Gaussian Processes for Data-Efficient Learning in Robotics and Control (2015, neutral): “X i Generally, model-based methods, i.e., methods which learn an explicit dynamics model of the environment, are more promising to efﬁciently extract valuable information from available data [5] than model-free methods, such as Q-learning [55] or TD-learning [52].”
A Non-Parametric Approach to Dynamic Programming (2011, background): “Model-based approaches are known to be more data efficient than direct methods, and lead to better policies [10, 11].”
PILCO: A Model-Based and Data-Efficient Approach to Policy Search (2011, background): “…why modelbased methods are not widely used in learning from scratch is that they suffer from model bias, i.e., they inherently assume that the learned dynamics model sufficiently accurately resembles the real environment, see, e.g., (Schneider, 1997; Schaal, 1997; Atkeson & Santamaŕıa, 1997).”
Convex Optimization: Algorithms and Complexity (2015, unknown): “There is some evidence that the first method provides better results with less data [Atkeson and Santamaria, 1997], and the second method may be more efficient in cases where the solution space (e.g., policy space) exhibits more regularity than the underlying dynamics, though there is some…”
Deep learning in robotics: a review of recent research (2017, methodology): “configuration is used in “model-based reinforcement learning” [46].”
Bayesian Time Series Models: Expectation maximisation methods for solving (PO)MDPs and optimal control problems (2011, unknown): “, (Atkeson and Santamaŕıa, 1997)).”
Deep-learning in Mobile Robotics - from Perception to Control Systems: A Survey on Why and Why not (2016, background): “Generalized mobile robotics research I.”
Information-Theoretic Motor Skill Learning (2013, methodology): “As model-based approaches are sensitive to model errors (Atkeson and Santamarı́a 1997), we use Gaussian Process models to represent the uncertainty of the learned model itself, and, hence, reduce the effect of model errors (Deisenroth, Rasmussen, and Fox 2011).”
Unsupervised Meta-Learning for Reinforcement Learning (2018, methodology): “Model-based RL methods (Deisenroth & Rasmussen, 2011; Chua et al., 2018; Srinivas et al., 2018; Nagabandi et al., 2018b; Finn & Levine, 2017b; Atkeson & Santamaria, 1997) use unsupervised experience to learn a dynamics model but do not learn how to efﬁciently use this model to explore to solve new…”
Transfer Reinforcement Learning with Shared Dynamics (2017, methodology): “with κ = 0) which corresponds to the method in (Atkeson and Santamaria 1997), and with a learning that would not use any transfer.”
Expectation-maximization Methods for Solving (po)mdps and Optimal Control Problems (2010, unknown): “In model-based Reinforcement Learning the transition and reward probabilities are estimated from experience (see, e.g., (Atkeson and Santamaría, 1997)).”