Curiosity driven reinforcement learning for motion planning on humanoids

By Mikhail Frank, Jürgen Leitner, Marijn F. Stollenga, Alexander Förster, J. Urgen Schmidhuber. 2014

95 citations recorded by the source corpus.

Most previous work on artificial curiosity (AC) and intrinsic motivation focuses on basic concepts and theory. Experimental results are generally limited to toy scenarios, such as navigation in a simulated maze, or control of a simple mechanical system with one or two degrees of freedom. To study AC in a more realistic setting, we embody a curious agent in the complex iCub humanoid robot. Our novel reinforcement learning (RL) framework consists of a state-of-the-art, low-level, reactive control layer, which controls the iCub while respecting constraints, and a high-level curious agent, which explores the iCub's state-action space through information gain maximization, learning a world model from experience, controlling the actual iCub hardware in real-time. To the best of our knowledge, this is the first ever embodied, curious agent for real-time motion planning on a humanoid. We demonstrate that it can learn compact Markov models to represent large regions of the iCub's configuration space, and that the iCub explores intelligently, showing interest in its physical constraints as well as in objects it finds in its environment.

Subfields: Cognitive Neuroscience, Genetics, Social Psychology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Genetics

In Social Psychology

Citations in the corpus

Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation (2016, background): “[9] demonstrate the effectiveness of artificial curiosity using information gain maximization in a humanoid robot.”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “This is generally achieved by defining a heuristic based on some measurement of surprise, information gain, compression, or empowerment (Schmidhuber, 1991; Klyubin et al., 2005; Kaplan and Oudeyer, 2006; Schmidhuber, 2008; Frank et al., 2014; Mohamed and Rezende, 2015; Bellemare et al., 2016; Houtho”
World Discovery Models (2019, unknown): “…& Sastry, 2017; Schmidhuber, 1991a; Lopes et al., 2012), and ﬁnally information gain (Shyam et al., 2018; Itti & Baldi, 2009; Little & Sommer, 2013; Frank et al., 2014; Houthooft et al., 2016). iv the last component is an RL algorithm that ﬁnds an optimal policy with respect to the intrinsic…”
Learning by Asking Questions (2017, background): “Exploratory learning centers on settings in which an agent explores the environment to acquire supervision [37, 50]; it has been studied in the context of, among others, computer games and navigation [28, 39], multi-user games [36], inverse kinematics [5], and motion planning for humanoids [14].”
A Survey on AI-Driven Digital Twins in Industry 4.0: Smart Manufacturing and Advanced Robotics (2021, background): “, a humanoid robot with high degrees of freedom (DOFs) [286].”
Intrinsic motivations and open-ended development in animals, humans, and robots: an overview (2014, background): “The term “intrin-sic motivation” was ﬁrst used to describe a “manipulation drive” hypothesized to explain why rhesus monkeys would engage with mechanical puzzles for long periods of time without receiving extrinsic rewards (Harlow et al., 1950).”