Self-Supervised Representation Learning by Rotation Feature Decoupling

By Zeyu Feng, Chang Xu, Dacheng Tao. 2019

215 citations recorded by the source corpus.

We introduce a self-supervised learning method that focuses on beneficial properties of representation and their abilities in generalizing to real-world tasks. The method incorporates rotation invariance into the feature learning framework, one of many good and well-studied properties of visual representation, which is rarely appreciated or exploited by previous deep convolutional neural network based self-supervised representation learning methods. Specifically, our model learns a split representation that contains both rotation related and unrelated parts. We train neural networks by jointly predicting image rotations and discriminating individual instances. In particular, our model decouples the rotation discrimination from instance discrimination, which allows us to improve the rotation prediction by mitigating the influence of rotation label noise, as well as discriminate instances without regard to image rotations. The resulting feature has a better generalization ability for more various tasks. Experimental results show that our model outperforms current state-of-the-art methods on standard self-supervised feature learning benchmarks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Self-Supervised Learning of Pretext-Invariant Representations (2019, methodology): “Similar to our work, recent methods also focus on invariance [47] or decoupling the pretext task [17] to learn representations.”
ScaleNet: An Unsupervised Representation Learning Method for Limited Information (2023, background): “[12] improved the rotation-prediction task by learning the split representation that contains rotation-related and unrelated features.”
Self-labelling via simultaneous clustering and representation learning (2019, methodology): “And ﬁnally, by adopting a hybrid approach, similar to (Feng et al., 2019), of training an AlexNet with 10 heads and one additional head for computing the RotNet loss, we ﬁnd further improvement.”
Audio-Visual Instance Discrimination with Cross-Modal Agreement (2020, background): “Contrastive Learning techniques use a contrastive loss [28] to learn representations either by predicting parts of the data [32, 33, 61], or discriminating between individual training instances [17, 19, 31, 34, 53, 86, 88, 94].”
Learning To Classify Images Without Labels (2020, methodology): “orrect itself as it gradually becomes more certain (see Figure 6). Pretext task. In addition to the instance discrimination task (NCE [49]) from before, we consider RotNet [15] and Feature Decoupling [14] as possible pretext tasks to mine the nearest neighbors. RotNet is trained to predict image rot”
Self-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works (2022, background): “prediction task to construct the signal [117], [118],”
Steering Self-Supervised Feature Learning Beyond Local Pixel Statistics (2020, methodology): “[14] showed that a combination of the rotation prediction task by Gidaris et al.”
Unsupervised Image Classification for Deep Representation Learning (2020, unknown): “w puzzle [29] 18.2 28.8 34.0 33.9 27.1 Counting [30] 18.0 30.6 34.3 32.5 25.7 RotNet [14] 18.8 31.7 38.7 38.2 36.5 AND [21] 15.6 27.0 35.9 39.7 37.9 AET [40] 19.3 35.4 44.0 43.6 42.4 RotNet+retrieval [13] 22.2 38.2 45.7 48.7 48.3 Ablation Study on Data Augmentation. Data augmentation plays an import”
Automatic Shortcut Removal for Self-Supervised Representation Learning (2020, background): “Another research direction is to combine several pretext tasks into one, which often improves performance (Doersch & Zisserman, 2017; Feng et al., 2019; Misra & van der Maaten, 2019).”
Multi-modal Self-Supervision from Generalized Data Transformations (2020, methodology): “We expect our method to yield even better results when combined with stronger architectures, as combining tasks has been shown to improve feature learning performance in multiple other papers [10, 67, 13].”
SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery (2022, background): “In recent years, self-supervised learning techniques have quickly become the norm for pre-training models on large-scale natural image datasets [1, 2, 3, 4, 5, 6, 7, 8], and have demonstrated strong performance on downstream tasks including image classification [3, 4, 9, 10], image segmentation [3, ”
SEED: Self-supervised Distillation For Visual Representation (2021, background): “Major ramifications of visual SSL include pretext tasks (Noroozi & Favaro, 2016; Zhang et al., 2016; Gidaris et al., 2018; Zhang et al., 2019; Feng et al., 2019), contrastive representation learning (Wu et al.”
Self-Supervised Learning in Remote Sensing: A review (2022, background): “Among others, predicting rotation angles [49], [128] and image inpainting [64], [129] are popular approaches.”
Charting the Right Manifold: Manifold Mixup for Few-shot Learning (2019, background): “Also, the authors of [13] proposed to decouple representation learning of the rotation as pretext task from class discrimination to obtain better results.”