Unsupervised object learning from dense equivariant image labelling

By James Thewlis, Hakan Bilen, Andrea Vedaldi. 2017

26 citations recorded by the source corpus.

One of the key challenges of visual perception is to extract abstract models of 3D objects and object categories from visual measurements, which are affected by complex nuisance factors such as viewpoint, occlusion, motion, and deformations. Starting from the recent idea of viewpoint factorization, we propose a new approach that, given a large number of images of an object and no other supervision, can extract a dense object-centric coordinate frame. This coordinate frame is invariant to deformations of the images and comes with a dense equivariant labelling neural network that can map image pixels to their corresponding object coordinates. We demonstrate the applicability of this method to simple articulated objects and deformable objects such as human faces, learning embeddings from random synthetic transformations or optical flow correspondences, all without any manual supervision.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

DensePose: Dense Human Pose Estimation in the Wild (2018, methodology): “More recently, [44] used the equivariance principle in order to align sets of images to a common coordinate system, while following the general idea of groupwise image alignment, e.”
GAN-Supervised Dense Visual Alignment (2021, background): “[77] James Thewlis, Andrea Vedaldi, and Hakan Bilen.”
Self-Supervised Learning of Geometrically Stable Features Through Probabilistic Introspection (2018, methodology): “Nevertheless, the method of [33] was found to be too fragile to work well in challenging data where significant occlusions may be present.”
Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance (2018, methodology): “In Table 2 we compare with the results of the best current method for semi-supervised image registration [29].”
Unsupervised Discovery of Object Landmarks as Structural Representations (2018, background): “[58] tried to densely map different objects to a canonical coordinate that reflects object structures.”
Unsupervised Learning of Object Landmarks through Conditional Image Generation (2018, methodology): “We follow [45, 44] and use unsupervised keypoints learnt on CelebA and VoxCeleb to regress manually-annotated keypoints in the MAFL and AFLW [23] test sets.”