Revisiting Feature Prediction for Learning Visual Representations from Video
By Adrien Bardes, Quentin Garrido, Jean Ponce, Xinlei Chen, Michael G. Rabbat, Yann LeCun, Mahmoud Assran, Nicolas Ballas. Trans. Mach. Learn. Res., 2024
396 citations recorded by the source corpus.
This paper explores feature prediction as a stand-alone objective for unsupervised learning from video and introduces V-JEPA, a collection of vision models trained solely using a feature prediction objective, without the use of pretrained image encoders, text, negative examples, reconstruction, or other sources of supervision. The models are trained on 2 million videos collected from public datasets and are evaluated on downstream image and video tasks. Our results show that learning by predicting video features leads to versatile visual representations that perform well on both motion and appearance-based tasks, without adaption of the model's parameters; e.g., using a frozen backbone. Our largest model, a ViT-H/16 trained only on videos, obtains 81.9% on Kinetics-400, 72.2% on Something-Something-v2, and 77.9% on ImageNet1K.
Subfields: Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/872d78c04c8fb115d492eea98199407991670533
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Jitendra Malik
- Andrea Vedaldi
- Xiaogang Wang
- Ping Luo
- Trevor Darrell
- Andrew Zisserman
Citations in the corpus
- UniVLA: Learning to Act Anywhere with Task-centric Latent Actions (2025, positive): “Inspired by joint-embedding predictive architectures (JEPA) [5, 6, 96], we propose using DINOv2 [62] spatial patch features as semantically rich representations.”
- V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning (2025, positive): “Models are ViT-L/16 trained for 90K iterations using a cosine learning schedule following Bardes et al. (2024).”
- Giving Sensors a Voice: Multimodal JEPA for Semantic Time-Series Embeddings (2026, unknown): “This observation is in line with (Bardes et al., 2024), which demonstrated that using attentive probing to pool embeddings was empirically superior for downstream task performance compared to directly mean pooling the embeddings, which can potentially result in lossy, diffuse representations which…”