Momentum Contrast for Unsupervised Visual Representation Learning

By Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, Ross B. Girshick. Computer Vision and Pattern Recognition, 2019

15,360 citations recorded by the source corpus.

We present Momentum Contrast (MoCo) for unsupervised visual representation learning. From a perspective on contrastive learning as dictionary look-up, we build a dynamic dictionary with a queue and a moving-averaged encoder. This enables building a large and consistent dictionary on-the-fly that facilitates contrastive unsupervised learning. MoCo provides competitive results under the common linear protocol on ImageNet classification. More importantly, the representations learned by MoCo transfer well to downstream tasks. MoCo can outperform its supervised pre-training counterpart in 7 detection/segmentation tasks on PASCAL VOC, COCO, and other datasets, sometimes surpassing it by large margins. This suggests that the gap between unsupervised and supervised representation learning has been largely closed in many vision tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

View Paper: https://arxiv.org/abs/1911.05722

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Yoshua Bengio
  4. Quoc V. Le
  5. Sergey Levine
  6. Percy Liang
  7. Trevor Darrell
  8. Dacheng Tao

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Ming-Hsuan Yang
  5. Xiaogang Wang
  6. Jitendra Malik
  7. Andrew Zisserman
  8. Yi Yang

In Representation & Self-Supervised Learning

  1. Trevor Darrell
  2. Ishan Misra
  3. Andrea Vedaldi
  4. Andrew Zisserman
  5. Yoshua Bengio
  6. Dacheng Tao
  7. Yann LeCun
  8. Abhinav Kumar Gupta

Citations in the corpus