Unsupervised Object Keypoint Learning using Local Spatial Predictability

By Anand Gopalakrishnan, Sjoerd van Steenkiste, J. Urgen Schmidhuber. 2020

21 citations recorded by the source corpus.

We propose PermaKey, a novel approach to representation learning based on object keypoints. It leverages the predictability of local image regions from spatial neighborhoods to identify salient regions that correspond to object parts, which are then converted to keypoints. Unlike prior approaches, it utilizes predictability to discover object keypoints, an intrinsic property of objects. This ensures that it does not overly bias keypoints to focus on characteristics that are not unique to objects, such as movement, shape, colour etc. We demonstrate the efficacy of PermaKey on Atari where it learns keypoints corresponding to the most salient object parts and is robust to certain visual distractors. Further, on downstream RL tasks in the Atari domain we demonstrate how agents equipped with our keypoints outperform those using competing alternatives, even on challenging environments with moving backgrounds or distractor objects.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

On the Binding Problem in Artificial Neural Networks (2020, background): “Recently, there has been renewed interest in saliency-based approaches through the discovery of keypoints (Jakab et al., 2018; Kulkarni et al., 2019; Minderer et al., 2019; Gopalakrishnan et al., 2020).”