Self-Supervised Learning of Geometrically Stable Features Through Probabilistic Introspection

By David Novotný, Samuel Albanie, Diane Larlus, Andrea Vedaldi. 2018

73 citations recorded by the source corpus.

Self-supervision can dramatically cut back the amount of manually-labelled data required to train deep neural networks. While self-supervision has usually been considered for tasks such as image classification, in this paper we aim at extending it to geometry-oriented tasks such as semantic matching and part detection. We do so by building on several recent ideas in unsupervised landmark detection. Our approach learns dense distinctive visual descriptors from an unlabeled dataset of images using synthetic image transformations. It does so by means of a robust probabilistic formulation that can introspectively determine which image regions are likely to result in stable image matching. We show empirically that a network pre-trained in this manner requires significantly less supervision to learn semantic object parts compared to numerous pre-training alternatives. We also show that the pre-trained representation is excellent for semantic object matching.

Subfields: Aerospace Engineering, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Self-Supervised Learning of Pretext-Invariant Representations (2019, background): “Although such covariance is beneficial for tasks such as predicting 3D correspondences [33, 57, 65], it is undesirable for most semantic recognition tasks.”
Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks (2018, background): “The features of this stage have been shown to generalise well to other semantic tasks [31].”
Lifelong Graph Learning (2020, neutral): “The training loss function is adapted from [29] which maximizes the likelihood of predicting similar node embeddings corresponding to their spatial location.”
Cross Pixel Optical Flow Similarity for Self-Supervised Learning (2018, positive): “[39] use correspondences obtained from synthetic warps to learn a dense image representation.”
From Coarse to Fine: Robust Hierarchical Localization at Large Scale (2018, background): “Dense pixel-wise features naturally emerge from CNNs and provide a powerful representation used for image matching [10, 13, 35, 38] and localization [49, 42].”
Self-supervised learning of a facial attribute embedding from video (2018, unknown): “ supervision from image data, the images themselves can be transformed via a synthetic warp or rotation and the network trained to recognise the rotation [19] or to learn equivariant pixel embeddings [40,51,52]. Of more direct relevance to our training framework are self-supervised frameworks that u”