Toward Self-Supervised Object Detection in Unlabeled Videos

By Elad Amrani, Rami Ben-Ari, Tal Hakim, A. Bronstein. 2019

5 citations recorded by the source corpus.

Unlabeled video in the wild presents a valuable, yet so far unharnessed, source of information for learning vision tasks. We present the first attempt of fully self-supervised learning of object detection from subtitled videos without any manual object annotation. To this end, we use the How2 multi-modal collection of instructional videos with English subtitles. We pose the problem as learning with a weakly- and noisily-labeled data, and propose a novel training model that can confront high noise levels, and yet train a classifier to localize the object of interest in the video frames, without any manual labeling involved. We evaluate our approach on a set of 11 manually annotated objects in over 5000 frames and compare it to an existing weakly-supervised approach as baseline. Benchmark data and code will be released upon acceptance of the paper.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

End-to-End Learning of Visual Representations From Uncurated Instructional Videos (2019, positive): “To avoid labeling visual data, several approaches have leveraged audio transcripts obtained from narrated videos using automatic speech recognition (ASR) as a way to supervise video models for object detection [4, 16, 56], captioning [34, 72], classification [3, 44, 49, 89], summarization [59] or re”
Grounding Object Detections With Transcriptions (2019, methodology): “There are four fundamental differences between (Amrani et al., 2019) and our work.”