Supplementary Material for Unsupervised Pre-Training of Image Features on Non-Curated Data

By Mathilde Caron, Piotr Bojanowski, Julien Mairal, Armand Joulin. 2019

Subfields: Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

Citations in the corpus

VirTex: Learning Visual Representations from Textual Annotations (2020, background): “Some recent approaches have begun to match or exceed supervised pretraining on ImageNet [22–26], and have been scaled to hundreds of millions [22, 24, 27, 28] or billions [25] of images.”
Self-Supervised Learning of Pretext-Invariant Representations (2019, methodology): “Following [7, 21], we randomly select a subset of 1 million images (YFCC-1M) from the 100 million images in YFCC.”
Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency (2022, background): “Hence, pre-training models in CV [36, 37, 14] and NLP [10, 15, 38] are not directly applicable due to data modality mismatch, and the existing results leave room for improvement [31, 39, 40].”
Source Data-Absent Unsupervised Domain Adaptation Through Hypothesis Transfer and Labeling Transfer (2020, background): “Some popular image-specific self-supervision tasks include example colorization [69], relative position prediction [70], rotation prediction [28], solving jigsaw puzzles [71]; on the other hand, contrastive losses [72], [73] and clustering losses [74], [75] focus on the similarity of sample pairs in”
Divide and Contrast: Self-supervised Learning from Uncurated Data (2021, methodology): “Another line of methods generates pseudo labels by clustering features [8, 9, 46, 105, 1].”
Efficient Visual Pretraining with Contrastive Detection (2021, background): “While the benefits of instance-discrimination approaches have mostly been limited to pretraining from simple datasets such as ImageNet, clustering-based pretraining has proven very successful in leveraging large amounts of uncurated images for transfer learning [3, 6, 7, 20, 31].”
Learning Representations by Predicting Bags of Visual Words (2020, methodology): “Our work presents similarities to the Deep(er) Clustering approach [8, 9].”