Learning Robot Skills with Temporal Variational Inference
By Tanmay Shankar, Abhinav Kumar Gupta. 2020
82 citations recorded by the source corpus.
In this paper, we address the discovery of robotic options from demonstrations in an unsupervised manner. Specifically, we present a framework to jointly learn low-level control policies and higher-level policies of how to use them from demonstrations of a robot performing various tasks. By representing options as continuous latent variables, we frame the problem of learning these options as latent variable inference. We then present a temporal formulation of variational inference based on a temporal factorization of trajectory likelihoods,that allows us to infer options in an unsupervised manner. We demonstrate the ability of our framework to learn such options across three robotic demonstration datasets.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Pieter Abbeel
- Chelsea Finn
- Shimon Whiteson
- Ofir Nachum
- Ruslan Salakhutdinov
- Aviral Kumar
- Yoshua Bengio
Citations in the corpus
- OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning (2020, methodology): “However, there have been some recent variants of the above works (Lynch et al., 2020; Shankar & Gupta, 2020; Krishnan et al., 2017; Merel et al., 2018) which extract skills from a prior dataset and using it for either performing tasks directly (Lynch et al., 2020) or learning downstream tasks…”
- RODE: Learning Roles to Decompose Multi-Agent Tasks (2020, background): “…et al., 2018; Sukhbaatar et al., 2018; Dwiel et al., 2019; Nair & Finn, 2019; Nasiriany et al., 2019; Dilokthanakul et al., 2019) or by discovering reusable skills (Daniel et al., 2012; Gregor et al., 2016; Warde-Farley et al., 2018; Shankar & Gupta, 2020; Thomas et al., 2018; Sharma et al., 2020).”
- Parrot: Data-Driven Behavioral Priors for Reinforcement Learning (2020, methodology): “Prior works in hierarchical imitation learning (Fox et al., 2017; Shankar & Gupta, 2020) train latent variable models over expert demonstrations to discover options, and later utilize these options to learn long-horizon tasks using RL.”
- Action and Perception as Divergence Minimization (2020, methodology): “…introduced information-based skill discovery as an extension of empowerment, motivating a line of work including SNN (Florensa et al., 2017), DIAYN (Eysenbach et al., 2018), work by Hausman et al. (2018), VALOR (Achiam et al., 2018), and work by Tirumala et al. (2019) and (Shankar and Gupta, 2020).”