The ImageNet Shuffle: Reorganized Pre-training for Video Event Detection

By Pascal Mettes, Dennis C. Koelma, Cees G. M. Snoek. 2016

123 citations recorded by the source corpus.

This paper strives for video event detection using a representation learned from deep convolutional neural networks. Different from the leading approaches, who all learn from the 1,000 classes defined in the ImageNet Large Scale Visual Recognition Challenge, we investigate how to leverage the complete ImageNet hierarchy for pre-training deep networks. To deal with the problems of over-specific classes and classes with few images, we introduce a bottom-up and top-down approach for reorganization of the ImageNet hierarchy based on all its 21,814 classes and more than 14 million images. Experiments on the TRECVID Multimedia Event Detection 2013 and 2015 datasets show that video representations derived from the layers of a deep neural network pre-trained with our reorganized hierarchy i) improves over standard pre-training, ii) is complementary among different reorganizations, iii) maintains the benefits of fusion with other modalities, and v) leads to state-of-the-art event detection results. The reorganized hierarchies and their derived Caffe models are publicly available at http://tinyurl.com/imagenetshuffle.

Subfields: Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100 (2020, methodology): “Increasingly, datasets have been used for novel tasks, through pre-training [10,11], self-supervision [12,13] or additional annotations [14,15].”
Temporal Context Network for Activity Localization in Videos (2017, methodology): “While classifying proposals, we also fuse two global video level priors using ImageNet shuffle features [19] and “actionness” features to further improve classification performance, as shown in [32].”
COCO-CN for Cross-Lingual Image Tagging, Captioning, and Retrieval (2018, methodology): “e model is pre-trained on the full ImageNet dataset with a boom-up reorganization [20] of the 22k ImageNet classes.”
Word2VisualVec: Image and Video to Sentence Matching by Visual Feature Prediction (2016, methodology): “We ex- periment with four pre-trained ConvNets, i.e., CaffeNet [14], GoogLeNet [31], ResNet-152 [10], and GoogLeNetshuffle [22].”
Predicting Visual Features From Text for Image and Video Caption Retrieval (2017, unknown): “GoogLeNet-shuffle [61] and ResNet-152 [19].”