MixCo: Mix-up Contrastive Learning for Visual Representation
By Sungnyun Kim, Gihun Lee, Sangmin Bae, Seyoung Yun. 2020
95 citations recorded by the source corpus.
Contrastive learning has shown remarkable results in recent self-supervised approaches for visual representation. By learning to contrast positive pairs' representation from the corresponding negatives pairs, one can train good visual representations without human annotations. This paper proposes Mix-up Contrast (MixCo), which extends the contrastive learning concept to semi-positives encoded from the mix-up of positive and negative images. MixCo aims to learn the relative similarity of representations, reflecting how much the mixed images have the original positives. We validate the efficacy of MixCo when applied to the recent self-supervised learning algorithms under the standard linear evaluation protocol on TinyImageNet, CIFAR10, and CIFAR100. In the experiments, MixCo consistently improves test accuracy. Remarkably, the improvement is more significant when the learning capacity (e.g., model size) is limited, suggesting that MixCo might be more useful in real-world scenarios. The code is available at: https://github.com/Lee-Gihun/MixCo-Mixup-Contrast.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Quoc V. Le
- Eric P. Xing
- Jiashi Feng
- Yoshua Bengio
- Mohit Bansal
- Trevor Darrell
- Dacheng Tao
- Jianfeng Gao
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Trevor Darrell
- Dacheng Tao
- Jiashi Feng
- Ming-Hsuan Yang
- Xiaogang Wang
- Alan L. Yuille
- Yi Yang
Citations in the corpus
- i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning (2021, result): “Kim et al. (2020) reported similar observations to ours but focused on small image datasets.”
- A Generalized Supervised Contrastive Learning Framework (2022, methodology): “Mixup and Cutmix can be applied to both contrasts and anchors because of generalized supervised contrastive loss, which gives stronger regularization than just applying to anchors [24].”
- Un-mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learning (2020, background): “(iii) Differences to i -Mix [Lee et al. 2021] and MixCo [Kim et al. 2020].”
- Towards Domain-Agnostic Contrastive Learning (2020, methodology): “…based contrastive learning methods such as ours and (Shen et al., 2020; Kalantidis et al., 2020; Kim et al., 2020b) have advantage over recently proposed adversarial direction based contrastive learning method (Kim et al., 2020a) because the later method requires additional gradient computation.”
- Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regularized Fine-Tuning (2021, methodology): “As an effective data augmentation method, mixup [70] has recently been applied to instance augmentation for CSL [24, 27, 32, 47].”