Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning

By Jean-Bastien Grill, Florian Strub, Florent Altch'e, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, B. '. Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar and 4 more. 2020

8,797 citations recorded by the source corpus.

We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other. From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view. At the same time, we update the target network with a slow-moving average of the online network. While state-of-the art methods rely on negative pairs, BYOL achieves a new state of the art without them. BYOL reaches $74.3\%$ top-1 classification accuracy on ImageNet using a linear evaluation with a ResNet-50 architecture and $79.6\%$ with a larger ResNet. We show that BYOL performs on par or better than the current state of the art on both transfer and semi-supervised benchmarks. Our implementation and pretrained models are given on GitHub.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Jianfeng Gao
  2. Yoshua Bengio
  3. Mohit Bansal
  4. Quoc V. Le
  5. Sergey Levine
  6. Dacheng Tao
  7. Trevor Darrell
  8. Percy Liang

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Trevor Darrell
  4. Ming-Hsuan Yang
  5. Xiaogang Wang
  6. Jitendra Malik
  7. Andrew Zisserman
  8. Yi Yang

In Representation & Self-Supervised Learning

  1. Trevor Darrell
  2. Yoshua Bengio
  3. Ishan Misra
  4. Yann LeCun
  5. Dacheng Tao
  6. Andrea Vedaldi
  7. Andrew Zisserman
  8. Ruslan Salakhutdinov

Citations in the corpus