OBoW: Online Bag-of-Visual-Words Generation for Self-Supervised Learning

By Spyros Gidaris, Andrei Bursuc, Gilles Puy, Nikos Komodakis, Matthieu Cord, Patrick Pérez. 2020

86 citations recorded by the source corpus.

Learning image representations without human supervision is an important and active research field. Several recent approaches have successfully leveraged the idea of making such a representation invariant under different types of perturbations, especially via contrastive-based instance discrimination training. Although effective visual representations should indeed exhibit such invariances, there are other important characteristics, such as encoding contextual reasoning skills, for which alternative reconstruction-based approaches might be better suited.With this in mind, we propose a teacher-student scheme to learn representations by training a convolutional net to reconstruct a bag-of-visual-words (BoW) representation of an image, given as input a perturbed version of that same image. Our strategy performs an online training of both the teacher network (whose role is to generate the BoW targets) and the student network (whose role is to learn representations), along with an online update of the visual-words vocabulary (used for the BoW targets). This idea effectively enables fully online BoW-guided unsupervised learning. Extensive experiments demonstrate the interest of our BoWbased strategy, which surpasses previous state-of-the-art methods (including contrastive-based ones) in several applications. For instance, in downstream tasks such Pascal object detection, Pascal classification and Places205 classification, our method improves over all prior unsupervised approaches, thus establishing new state-of-the-art results that are also significantly better even than those of supervised pre-training. We provide the implementation code at https://github.com/valeoai/obow.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Qi Tian
2. Lingxi Xie
3. Guo-Jun Qi
4. Simone Scardapane
5. Valerio Marsocci
6. Xiaopeng Zhang
7. Xiao Wang
8. H. Xiong

In Computer Vision and Pattern Recognition

1. Qi Tian
2. Lingxi Xie
3. Dacheng Tao
4. Yi Yang
5. Alan L. Yuille
6. Guo-Jun Qi
7. Luc Van Gool
8. Xiaopeng Zhang

Citations in the corpus

• Continual Barlow Twins: Continual Self-Supervised Learning for Remote Sensing Semantic Segmentation (2022, methodology): “and BT was based on the current RS literature on SSL [11], [13], [20], [21], [22], and CV more in general [7], [8], [16], [23].”
• Contrastive Learning With Stronger Augmentations (2021, methodology): “Also, BoWNet [53] and OBOW [54] even included CutMix [55] as stronger augmentation to contribute”
• Seed the Views: Hierarchical Semantic Alignment for Contrastive Representation Learning (2022, background): “Besides, BoWNet [35] and OBoW [36] also use the vanilla CutMix by mixing two randomly sampled images.”
• MARE: Self-Supervised Multi-Attention REsu-Net for Semantic Segmentation in Remote Sensing (2021, methodology): “Most of them trace the one presented in [12,18].”