Unsupervised Learning of Object Keypoints for Perception and Control

By Tejas D. Kulkarni, Ankush Gupta, Catalin Ionescu, Sebastian Borgeaud, Malcolm Reynolds, Andrew Zisserman, Volodymyr Mnih. 2019

218 citations recorded by the source corpus.

The study of object representations in computer vision has primarily focused on developing representations that are useful for image classification, object detection, or semantic segmentation as downstream tasks. In this work we aim to learn object representations that are useful for control and reinforcement learning (RL). To this end, we introduce Transporter, a neural network architecture for discovering concise geometric object representations in terms of keypoints or image-space coordinates. Our method learns from raw video frames in a fully unsupervised manner, by transporting learnt image features between video frames using a keypoint bottleneck. The discovered keypoints track objects and object parts across long time-horizons more accurately than recent similar methods. Furthermore, consistent long-term tracking enables two notable results in control domains -- (1) using the keypoint co-ordinates and corresponding image features as inputs enables highly sample-efficient reinforcement learning; (2) learning to explore by controlling keypoint locations drastically reduces the search space, enabling deep exploration (leading to states unreachable through random action exploration) without any extrinsic rewards.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Reinforcement Learning

Citations in the corpus

Object-Centric Learning with Slot Attention (2020, background): “Obtaining object-centric representations from raw perceptual input, such as an image or a video, is challenging and often requires either supervision [1, 3, 9, 10] or task-specific architectures [2, 11].”
Toward Causal Representation Learning (2021, background): “Machine learning models that incorporate or learn structural knowledge of an environment have been shown to be more efficient and generalize better [9], [11], [15], [16], [27], [58], [77], [84], [85], [141], [157], [177], [181], [197], [211], [212], [244], [258], [272], [274].”
On the Binding Problem in Artificial Neural Networks (2020, background): “Recently, there has been renewed interest in saliency-based approaches through the discovery of keypoints (Jakab et al., 2018; Kulkarni et al., 2019; Minderer et al., 2019; Gopalakrishnan et al., 2020).”
Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control (2021, background): “Keypoints have been shown to be an effective representation for visuomotor policy learning [8, 14, 21, 22, 26].”
Transporter Networks: Rearranging the Visual World for Robotic Manipulation (2020, background): “Alternative representations including keypoints [3, 4, 5, 6] and dense descriptors [9, 10, 11] have shown to be capable of category-level generalization and representing deformable objects, but still struggle to represent scenes with an unknown number of objects (e.”
Self-Supervised Correspondence in Visuomotor Policy Learning (2019, background): “Relevant concurrent works include [19] which proposes autoencoder-style visual training but with a reference image and novel architecture, and [20] which proposes a graphbased reward function using a fixed set of correspondences.”
CLIPort: What and Where Pathways for Robotic Manipulation (2021, background): “Alternatively, dense descriptors [27, 28, 29] and keypoint representations [30, 31, 32] forgo segmentation and pose representations, but do not reason about sequential actions and struggle to represent scenes with variable numbers of objects.”
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor With Application to In-Hand Manipulation (2020, methodology): “predictive model, based on the “Structural VRNN” architecture proposed in [31], which is also closely related to approaches proposed in [32], [33].”
Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learning (2020, methodology): “On task top-down camera , the transporter [14] model was able to achieve performance that was only slightly worse than WDS and SDS , while the autoencoder performed signiﬁcantly worse.”
Unsupervised Object Keypoint Learning using Local Spatial Predictability (2020, methodology): “We observe that agents using PermaKey keypoints (PKey-CNN in Table 1) outperform corresponding variants using Transporter keypoints (original from Kulkarni et al. (2019) and our re-implementation) on 3 environments1.”
Entity Abstraction in Visual Model-Based Reinforcement Learning (2019, background): “The KeyNet encoder of the Transporter, the monolithic dynamics model of COBRA, and the CNN-encoder of C-SWMs all by their nature associate the content of the entity representation with the index of that representation (Figure 1d), thereby violating the entity abstraction.”
Investigating Simple Object Representations in Model-Free Deep Reinforcement Learning (2020, unknown): “More recently, this inductive bias served as a source of inspiration for several recent advancements in deep learning, such as Kulkarni et al. (2016), van Steenkiste et al. (2018), Kulkarni et al. (2019), Veerapaneni et al. (2019), and Lin et al. (2020).”
Neuroevolution of self-interpretable agents (2020, result): “Conceptually, our approach is similar to Spatial Softmax [26, 57, 96],”
Unsupervised Learning of Object Structure and Dynamics from Videos (2019, result): “Concurrent work that applies a similar keypoint-structured model to control tasks confirms these results [13].”