Learning object class detectors from weakly annotated video

By Alessandro Prest, Christian Leistner, Javier Civera, Cordelia Schmid, Vittorio Ferrari. 2012

498 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

MERLOT: Multimodal Neural Script Knowledge Models (2021, methodology): “Prior works have used web videos+ASR to build weakly-supervised object detectors [75], action detectors/classifiers [108, 5, 51, 72], instruction aligners [65, 4, 15], video captioners [85, 38, 74, 89], and visual reference resolvers [41].”
Extreme Clicking for Efficient Object Annotation (2017, background): “Training object class detectors from videos could bypass the need for manual bounding boxes, as the motion of the objects facilitates their automatic localization [49, 40, 41].”
Improving the Robustness of Deep Neural Networks via Stability Training (2016, background): “Finding such hard positives in video data for data augmentation has been used in [5, 4, 8] and has been found to improve predictive performance and consistency.”
Wanderlust: Online Continual Object Detection in the Real World (2021, background): “[23] extend the evaluation from static images to video snippets like Activity Daily Living (ADL) [33] dataset and YouTube Objects (YTO) datasets [34] for incremental domain adaptation.”
Unsupervised Learning of Edges (2015, methodology): “Video Dataset: For training, we combine videos from two different datasets: the Video Segmentation Benchmark (VSB) [16] and the YouTube Object dataset [28].”
Large-Scale Weakly-Supervised Pre-Training for Video Action Recognition (2019, background): “Learning from weak supervision: Given the known challenges in gathering exhaustive annotations for various image and video tasks, leveraging object labels and other meta information to supply weak supervision [7, 15, 19, 22, 30, 31, 39, 40, 50, 52, 53, 54, 55, 56, 59] is a widely-adopted approach.”
Computational Baby Learning (2014, background): “For the second category, video-based approaches [19][29][1] utilized intrinsic motion cues and appearance correlations within temporal adjacent frames to augment the model training.”
Cross Pixel Optical Flow Similarity for Self-Supervised Learning (2018, positive): “f-supervision task. We use early stopping to reduce over-tting by monitoring the loss on a validation set. The validation set consists of 5000 imageow pairs computed from the YouTube objects dataset [50]. Epic-Flow [51] + Deep-Matching [52] was used to compute opticalow for these frames. 4.3 Learnin”
Watch and learn: Semi-supervised learning of object detectors from videos (2015, methodology): “Simply put, we ﬁrst train detectors on annotated objects, followed by detection on input videos.”
Weakly Supervised Object Localization with Multi-Fold Multiple Instance Learning (2015, neutral): “Results for Pandey and Lazebnik [35] are taken from [37].”
Unsupervised Object Discovery and Tracking in Video Collections (2015, methodology): “Localization accuracy is measured using CorLoc [17, 25, 27], which is deﬁned as the percentage of images correctly localized according to the P ASCAL criterion: area ( r p ∩ r gt ) area ( r p ∪ r gt ) > 0 .”
YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video (2017, methodology): “• The YouTube-Objects data set [38, 39] consists of a number of frames queried from YouTube with a few hundred curated bounding box annotations.”
Image Segmentation Using Deep Learning: A Survey (2020, background): “Youtube-Objects [158] contains videos collected from YouTube, which include objects from ten PASCAL VOC classes (aeroplane, bird, boat, car, cat, cow, dog, horse, motorbike, and train).”
One-Shot Video Object Segmentation (2016, methodology): “For completeness, we also experiment on two legacy databases: Youtube-objects [41], which was manually segmented by Jain and Grauman [20]; and SegTrack v2 [25].”
Learning Video Object Segmentation from Static Images (2016, methodology): “As mentioned in §5.3 in the main paper by adding additional ingredients speciﬁcally tuned for different datasets, such as optical ﬂow and CRF post-processing, we can push the results even further, reaching 80.3 mIoU on DAVIS, 72.6 on YoutubeObjects and 70.3 on SegTrackv-2.”
Track and Transfer: Watching Videos to Simulate Strong Human Supervision for Weakly-Supervised Object Detection (2016, methodology): “While some work consider learning object category models directly from (noisy) internet videos [28, 14, 23], we are exploring a rather different problem: we use video data to simulate human annotations, but ultimately use image data to train our models.”
Revisiting Knowledge Transfer for Training Object Class Detectors (2017, methodology): “Our scheme improves over previous usage of manually engineered object proposals and their associated objectness score [1, 10] in WSOL in two ways: (1) by using trained proposals and objectness scoring function trained on source classes; (2) generalizing the common use of a single classagnostic objec”
Training Object Class Detectors from Eye Tracking Data (2014, background): “The results reveal several interesting observations: (1) all feature types we propose contribute to the overall performance of the full model, as adding each type in turn progressively leads to higher CorLoc; (2) with 49.6% CorLoc, our full model significantly outperforms all baselines, including re”
Continual egocentric object recognition (2019, background): “Indeed, recent works [16, 12] have shown how unsupervised pre-training with videos enables to learn useful representations for image classiﬁcation tasks, and how few videos of a given class of objects are sufﬁcient to train a detector for that class [22].”