Self-Supervised Visual Representation Learning from Hierarchical Grouping
By Xiao Zhang, Michael Maire. 2020
80 citations recorded by the source corpus.
We create a framework for bootstrapping visual representation learning from a primitive visual grouping capability. We operationalize grouping via a contour detector that partitions an image into regions, followed by merging of those regions into a tree hierarchy. A small supervised dataset suffices for training this grouping primitive. Across a large unlabeled dataset, we apply this learned primitive to automatically predict hierarchical region structure. These predictions serve as guidance for self-supervised contrastive feature learning: we task a deep network with producing per-pixel embeddings whose pairwise distances respect the region hierarchy. Experiments demonstrate that our approach can serve as state-of-the-art generic pre-training, benefiting downstream tasks. We additionally explore applications to semantic region search and video-based object instance tracking.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Du Tran
- Andrew Zisserman
- Jitendra Malik
- Cordelia Schmid
- Lorenzo Torresani
- Trevor Darrell
- Christoph Feichtenhofer
- Mohit Bansal
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Andrew Zisserman
- Jitendra Malik
- Cordelia Schmid
- Dacheng Tao
- Trevor Darrell
- Xiaogang Wang
- Andrea Vedaldi
In Representation & Self-Supervised Learning
- Aäron van den Oord
- Yonglong Tian
- Andrew Zisserman
- Jitendra Malik
- Jean-Baptiste Alayrac
- Olivier J. H'enaff
- Phillip Isola
- Oriol Vinyals
Citations in the corpus
- Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity (2022, methodology): “In segmentation literature [3, 60], Oriented Watershed Transform (OWT), globalized contour through Normalized Cut (gPb) [48] and Ultrametric Contour Map (UCM) [2] are also used for grouping from the edge map of an image.”
- Divide and Contrast: Self-supervised Learning from Uncurated Data (2021, background): “Such views can be created by color decomposition [88], patch cropping [69, 43, 3], data augmentation [12, 13, 83], or image segmentation [42, 92, 109].”
- Efficient Visual Pretraining with Contrastive Detection (2021, background): “Most related to our work are [59, 68] that also leverage image segmentations for self-supervised learning, although both differ from ours in that they learn backbones that are specialized for semantic segmentation and employ different loss functions.”