Few-Shot Video Classification via Temporal Alignment

By Kaidi Cao, Jingwei Ji, Zhangjie Cao, Chien-Yi Chang, Juan Carlos Niebles. 2019

294 citations recorded by the source corpus.

Difficulty in collecting and annotating large-scale video data raises a growing interest in learning models which can recognize novel classes with only a few training examples. In this paper, we propose the Ordered Temporal Alignment Module (OTAM), a novel few-shot learning framework that can learn to classify a previously unseen video. While most previous work neglects long-term temporal ordering information, our proposed model explicitly leverages the temporal ordering information in video data through ordered temporal alignment. This leads to strong data-efficiency for few-shot learning. In concrete, our proposed pipeline learns a deep distance measurement of the query video with respect to novel class proxies over its alignment path. We adopt an episode-based training scheme and directly optimize the few-shot learning objective. We evaluate OTAM on two challenging real-world datasets, Kinetics and Something-Something-V2, and show that our model leads to significant improvement of few-shot video classification over a wide range of competitive baselines and outperforms state-of-the-art benchmarks by a large margin.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks (2022, background): “, object detection [64], semantic learning [65], video classification [66] and face alignment [67].”
When Video Classification Meets Incremental Classes (2021, background): “With the rise of deep learning, CNNs dominated the video understanding literature recently [3, 21, 27, 30, 36, 44, 54, 56], instead of hand-crafted features [1, 26, 35, 46, 47].”
Generalized Many-Way Few-Shot Video Classification (2020, methodology): “While the support set is used to adapt the model to recognize novel classes, the classiﬁcation of [48] and [2] on Kinetics and SomethingV2 respectively.”
Human Action Recognition and Prediction: A Survey (2018, methodology): “Similar toARN,Ordered Temporal AlignmentModule (OTAM) (Cao et al., 2020) extracts per-frame feature through an embedding network, then computes an alignment score of the distance matrix.”