Fast Model Identification via Physics Engines for Data-Efficient Policy Search

By Shaojun Zhu, Andrew Kimmel, Kostas E. Bekris, Abdeslam Boularias. 2017

53 citations recorded by the source corpus.

This paper presents a method for identifying mechanical parameters of robots or objects, such as their mass and friction coefficients. Key features are the use of off-the-shelf physics engines and the adaptation of a Bayesian optimization technique towards minimizing the number of real-world experiments needed for model-based reinforcement learning. The proposed framework reproduces in a physics engine experiments performed on a real robot and optimizes the model's mechanical parameters so as to match real-world trajectories. The optimized model is then used for learning a policy in simulation, before real-world deployment. It is well understood, however, that it is hard to exactly reproduce real trajectories in simulation. Moreover, a near-optimal policy can be frequently found with an imperfect model. Therefore, this work proposes a strategy for identifying a model that is just good enough to approximate the value of a locally optimal policy with a certain confidence, instead of wasting effort on identifying the most accurate model. Evaluations, performed both in simulation and on a real robotic manipulation task, indicate that the proposed strategy results in an overall time-efficient, integrated model identification and learning solution, which significantly improves the data-efficiency of existing policy search algorithms.

Subfields: Artificial Intelligence, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

Citations in the corpus

Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience (2018, methodology): “[31] suggests to optimize the simulation parameters such that the value function is well approximated in simulation without replicating the real world dynamics.”
Block Contextual MDPs for Continual Learning (2021, background): “Our work is related to four broad areas: (i) System Identification and Adaptive Control, (ii) Continual RL, (iii) Context Modeling and (iv) Meta RL and Multitask RL. System Identification and Adaptive Control (Zadeh, 1956; Åström & Bohlin, 1965; Swevers et al., 1997; Bhat et al., 2002; Gevers et al.”