Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset

By João Carreira, Andrew Zisserman. 2017

9,734 citations recorded by the source corpus.

The paucity of videos in current action classification datasets (UCF-101 and HMDB-51) has made it difficult to identify good video architectures, as most methods obtain similar performance on existing small-scale benchmarks. This paper re-evaluates state-of-the-art architectures in light of the new Kinetics Human Action Video dataset. Kinetics has two orders of magnitude more data, with 400 human action classes and over 400 clips per class, and is collected from realistic, challenging YouTube videos. We provide an analysis on how current architectures fare on the task of action classification on this dataset and how much performance improves on the smaller benchmark datasets after pre-training on Kinetics. We also introduce a new Two-Stream Inflated 3D ConvNet (I3D) that is based on 2D ConvNet inflation: filters and pooling kernels of very deep image classification ConvNets are expanded into 3D, making it possible to learn seamless spatio-temporal feature extractors from video while leveraging successful ImageNet architecture designs and even their parameters. We show that, after pre-training on Kinetics, I3D models considerably improve upon the state-of-the-art in action classification, reaching 80.2% on HMDB-51 and 97.9% on UCF-101.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Endocrinology, Diabetes and Metabolism

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Trevor Darrell
  4. Quoc V. Le
  5. Yoshua Bengio
  6. Dacheng Tao
  7. Sergey Levine
  8. Percy Liang

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Ming-Hsuan Yang
  4. Trevor Darrell
  5. Xiaogang Wang
  6. Jitendra Malik
  7. Andrew Zisserman
  8. Cordelia Schmid

In Endocrinology, Diabetes and Metabolism

  1. Christoph Feichtenhofer
  2. Limin Wang
  3. Dima Damen
  4. Luc Van Gool
  5. Lorenzo Torresani
  6. Cordelia Schmid
  7. Rohit Girdhar
  8. Yu Qiao

Citations in the corpus