A Cookbook of Self-Supervised Learning
By Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari S. Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Grégoire Mialon, Yuandong Tian and 9 more. arXiv.org, 2023
398 citations recorded by the source corpus.
Self-supervised learning, dubbed the dark matter of intelligence, is a promising path to advance machine learning. Yet, much like cooking, training SSL methods is a delicate art with a high barrier to entry. While many components are familiar, successfully training a SSL method involves a dizzying set of choices from the pretext tasks to training hyper-parameters. Our goal is to lower the barrier to entry into SSL research by laying the foundations and latest SSL recipes in the style of a cookbook. We hope to empower the curious researcher to navigate the terrain of methods, understand the role of the various knobs, and gain the know-how required to explore how delicious SSL can be.
Subfields: Artificial Intelligence
View Paper: https://www.semanticscholar.org/paper/6bfafb32b423c3f0456a10984814f89046def489
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yossi Adi
- Gabriel Synnaeve
- Jade Copet
- Alexandre D'efossez
- Tal Remez
- Felix Kreuk
- Itai Gat
- Philipp Hacker
Citations in the corpus
- Regulating ChatGPT and other Large Generative AI Models (2023, background): “[28, 53] that aim at allowing them “to find patterns and relationships”
- Simple and Controllable Music Generation (2023, background): “Recent advancements in self-supervised audio representation learning [Balestriero et al., 2023], sequential modeling [Touvron et al., 2023], and audio synthesis [Tan et al., 2021] provide the conditions to develop such models.”