Unsupervised Object Discovery and Tracking in Video Collections

By Suha Kwak, Minsu Cho, Ivan Laptev, Jean Ponce, Cordelia Schmid. 2015

127 citations recorded by the source corpus.

This paper addresses the problem of automatically localizing dominant objects as spatio-temporal tubes in a noisy collection of videos with minimal or even no supervision. We formulate the problem as a combination of two complementary processes: discovery and tracking. The first one establishes correspondences between prominent regions across videos, and the second one associates similar object regions within the same video. Interestingly, our algorithm also discovers the implicit topology of frames associated with instances of the same object class across different videos, a role normally left to supervisory information in the form of class labels in conventional image and video understanding methods. Indeed, as demonstrated by our experiments, our method can handle video collections featuring multiple object classes, and substantially outperforms the state of the art in colocalization, even though it tackles a broader problem with much less supervision.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning Open-World Object Proposals without Learning to Classify (2021, background): “They have become the integral pre-processing steps for many computer vision systems, including object detection [33,41,42,45,59,74], segmentation [3,8,13], object discovery [12, 16, 47], weakly supervised object detection [5, 21, 53], visual tracking [35, 63], content-aware retargeting [51], etc.”
Object-Proposal Evaluation Protocol is ‘Gameable’ (2015, background): “Object proposal algorithms have quickly become the defacto pre-processing step in a number of vision pipelines – object detection [12–21], segmentation [22–26], object discovery [27–30], weakly supervised learning of object-object interactions [31, 32], content aware media retargeting [33], action r”
Grounding of Textual Phrases in Images by Reconstruction (2015, background): “[21] address unsupervised object localization of dominant objects in videos by combining the problem with tracking.”
Detect to Track and Track to Detect (2017, unknown): “[15, 20].”
Discover and Learn New Objects from Documentaries (2017, background): “[20] formulate the problem of unsupervised spatiotemporal object localization as a combination of discovery and tracking.”
ContextLocNet: Context-Aware Deep Network Models for Weakly Supervised Localization (2016, background): “[31, 32] propose a contrast-based contextual score for unsupervised object localization, which measures the contrast of matching scores between a candidate region and its surrounding candidate regions.”
Unsupervised Image Matching and Object Discovery as Optimization (2019, unknown): “[12] (see [28] for related”
Learning About Objects by Learning to Interact with Them (2020, methodology): “The supervised video object discovery approaches use different forms of supervision such as CNNs pre-trained for other tasks [35, 58, 57], supervised object proposal models [19, 11, 34], fixation data as annotation [61], or instance segmentation for single frames [46].”
T-CNN: Tubelets With Convolutional Neural Networks for Object Detection From Videos (2016, background): “There have also been works on object localization and co-localization [18], [19], [20], [21].”
Object Detection from Video Tubelets with Convolutional Neural Networks (2016, methodology): “When these methods are applied to videos, they might miss some positive samples because the objects might not be of their best poses in each frame of the videos.”
STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos (2020, background): “Video Segmentation: Temporally consistent object segmentation in videos can benefit several other tasks such as action/activity recognition [29,35], video object detection [42,25] and object discovery [45,66,87,96,17,97].”
Spot On: Action Localization from Pointly-Supervised Proposals (2016, background): “Semi-supervised learning for video object detection is done with a few bounding boxes [40,41], a few global frame labels [42], only video class labels [43], or no labels at all [44].”
Weakly-Supervised Video Object Grounding from Text by Loss Weighting and Object Interaction (2018, methodology): “Weakly-supervised object localization has been explored in both the image [2, 4, 5, 14, 21] and the video domain [10, 17].”