Deep Object-Centric Representations for Generalizable Robot Learning

By Coline Devin, Pieter Abbeel, Trevor Darrell, Sergey Levine. 2017

127 citations recorded by the source corpus.

Robotic manipulation in complex open-world scenarios requires both reliable physical manipulation skills and effective and generalizable perception. In this paper, we propose using an object-centric prior and a semantic feature space for the perception system of a learned policy. We devise an object-level attentional mechanism that can be used to determine relevant objects from a few trajectories or demonstrations, and then immediately incorporate those objects into a learned policy. A task-independent attention locates possible objects in the scene, and a task-specific attention identifies which objects are predictive of the trajectories. The scope of the task-specific attention is easily adjusted by showing demonstrations with distractor objects or with diverse relevant objects. Our results indicate that this approach exhibits good generalization across object instances using very few samples, and can be used to learn a variety of manipulation tasks using reinforcement learning.

Subfields: Artificial Intelligence, Control and Systems Engineering, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Reinforcement Learning

Citations in the corpus

Object-Centric Learning with Slot Attention (2020, background): “Obtaining object-centric representations from raw perceptual input, such as an image or a video, is challenging and often requires either supervision [1, 3, 9, 10] or task-specific architectures [2, 11].”
Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation (2018, neutral): “Prior work has incorporated detectors into reinforcement learning-based settings [8], but for training a model-free policy that is only designed to generalize to new instances of the same event cues.”
Reasoning About Physical Interactions with Object-Oriented Prediction and Planning (2018, background): “Object-centric scene decompositions have also been considered as a potential state representation in reinforcement learning (Diuk et al., 2008; Scholz et al., 2014; Devin et al., 2017; Goel et al., 2018; Keramati et al., 2018).”
Modeling Long-horizon Tasks as Sequential Interaction Landscapes (2020, background): “Object-centric representations, learned from visual data, can serve as a powerful means to understand physical agent and object interactions [62, 10].”
Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies (2018, background): “They leverage the data either by making some task-agnostic assumption about the data distribution being simpler than the raw pixels (unsupervised approaches like the autoencoder family [34, 83, 41, 48] and Generative Adversarial Networks [77, 23, 57, 15]) or they exploit some known structure (so-cal”
Decoupling Representation Learning from Reinforcement Learning (2020, methodology): “(Devin et al., 2018; Kipf et al., 2019) pretrained object-centric representations, the latter learning a forward model by way of contrastive losses; (Yan et al.”
How to Close Sim-Real Gap? Transfer with Segmentation! (2020, unknown): “Devin et al. [2018], Singh et al. [2017] used network structures more similar to our work, where the network has a perception part and”
Reward Learning from Narrated Demonstrations (2018, background): “Some recent works do also consider object-centric input [14, 16].”
“Good Robot!”: Efficient Reinforcement Learning for Multi-Step Visual Tasks with Sim to Real Transfer (2019, unknown): “[13], [14].”
Graph-Structured Visual Imitation (2019, methodology): “[35] uses pretrained object detectors and learns attention over the obtained detection boxes, which are incorporated as part of the state representation for policy learning.”
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms (2019, background): “If the set of object properties is sufficiently rich, then the robot may only need to select a suitable set of these as features for learning [89, 223, 268, 350, 359].”
The Ingredients of Real-World Robotic Reinforcement Learning (2020, background): “…tackling this challenge have been discussed in prior work: design complete computer vision systems to detect objects and extract the reward signals (Devin et al., 2018; Nagabandi et al., 2019), engineer reward functions that use various task-speciﬁc heuristics to obtain rewards from pixels…”
Deep Object-Centric Policies for Autonomous Driving (2018, unknown): “Prior work has encoded objects as bounding box positions [14] for manipulation tasks, but does not use end-to-end training and discards the features of the objects, instead just concatenating their pixel positions.”