Evolving Losses for Unlabeled Video Representation Learning
By A. Piergiovanni, Anelia Angelova, Michael S. Ryoo. 2019
7 citations recorded by the source corpus.
We present a new method to learn video representations from unlabeled data. Given large-scale unlabeled video data, the objective is to benefit from such data by learning a generic and transferable representation space that can be directly used for a new task such as zero/few-shot learning. We formulate our unsupervised representation learning as a multi-modal, multi-task learning problem, where the representations are also shared across different modalities via distillation. Further, we also introduce the concept of finding a better loss function to train such multi-task multi-modal representation space using an evolutionary algorithm; our method automatically searches over different combinations of loss functions capturing multiple (self-supervised) tasks and modalities. Our formulation allows for the distillation of audio, optical flow and temporal information into a single, RGB-based convolutional neural network. We also compare the effects of using additional unlabeled video data and evaluate our representation learning on standard public video datasets.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yonglong Tian
- Dilip Krishnan
- Phillip Isola
- Jiashi Feng
- Ming-Hsuan Yang
- Andrew Zisserman
- Stefano Ermon
- Ziwei Liu
In Computer Vision and Pattern Recognition
- Yonglong Tian
- Luc Van Gool
- Phillip Isola
- Ming-Hsuan Yang
- Dacheng Tao
- Xiaogang Wang
- Trevor Darrell
- Andrea Vedaldi
In Representation & Self-Supervised Learning
- Phillip Isola
- Yonglong Tian
- Dilip Krishnan
- Ishan Misra
- Stefano Ermon
- Kai Han
- Daniel L. K. Yamins
- Bingchen Zhao
Citations in the corpus
- Contrastive Multiview Coding (2019, background): “The idea of contrastive learning has started to spread over many other tasks in various other domains [73,86,60,76,47,37,74,81].”
- Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction (2022, unknown): “Piergiovanni et al. (2020) learns the visual representation through a multi-tasking framework that incorporates a series of pretext tasks such as reconstruction and temporal ordering prediction.”