Progressive Teacher-Student Learning for Early Action Prediction

By Xionghui Wang, Jianfang Hu, J. Lai, Jianguo Zhang, Weishi Zheng. 2019

137 citations recorded by the source corpus.

The goal of early action prediction is to recognize actions from partially observed videos with incomplete action executions, which is quite different from action recognition. Predicting early actions is very challenging since the partially observed videos do not contain enough action information for recognition. In this paper, we aim at improving early action prediction by proposing a novel teacher-student learning framework. Our framework involves a teacher model for recognizing actions from full videos, a student model for predicting early actions from partial videos, and a teacher-student learning block for distilling progressive knowledge from teacher to student, crossing different tasks. Extensive experiments on three public action datasets show that the proposed progressive teacher-student learning framework can consistently improve performance of early action prediction model. We have also reported the state-of-the-art performances for early action prediction on all of these sets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Endocrinology, Diabetes and Metabolism

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Dacheng Tao
2. Qi Tian
3. Jun Liu
4. Li Fei-Fei
5. Yu Qiao
6. Hossein Rahmani
7. Qiuhong Ke
8. Juan Carlos Niebles

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Luc Van Gool
3. Yi Yang
4. Xiaogang Wang
5. Qi Tian
6. Alan L. Yuille
7. Li Fei-Fei
8. Ming-Hsuan Yang

In Endocrinology, Diabetes and Metabolism

1. Jun Liu
2. Juan Carlos Niebles
3. Qiuhong Ke
4. Yu Qiao
5. Bolei Zhou
6. Junjie Yan
7. Hossein Rahmani
8. Mengmeng Wang

Citations in the corpus

• Knowledge Distillation: A Survey (2020, methodology): “…extended to other visual recognition applications, including face recognition (Luo et al. 2016; Kong et al. 2019; Yan et al. 2019; Ge et al. 2018; Wang et al. 2018b, 2019c; Duong et al. 2019; Wu et al. 2020; Wang et al. 2017; Zhang et al. 2020b; Wang et al. 2020b), image/video segmentation (He…”
• Creating Something From Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing (2020, background): “Recently, many efforts were made to study what the dark knowledge is [41, 40], and/or how to efficiently take advantage of such knowledge [11, 42, 35, 2].”
• Procedure Planning in Instructional Videos (2019, background): “The others ask the question “What is going to happen?”, such as early action recognition [9, 35], action label prediction [1, 6, 9, 14, 22, 27, 29, 36], video prediction [18–20, 24, 26, 34], etc.”
• Else-Net: Elastic Semantic Network for Continual Action Recognition from Skeleton Data (2021, background): “Skeleton-based human action recognition has been attracting increasing attention in recent years because of its research significance [5, 32, 29] and relevance to a wide range of applications, e.”
• PlaTe: Visually-Grounded Planning With Transformers in Procedural Tasks (2021, background): “The other works [24] perform egocentric action anticipation model the relationships between past, future events, and incomplete observations.”
• Collaborative Distillation in the Parameter and Spectrum Domains for Video Action Recognition (2020, background): “(Lin, Xiao, and Fan 2018; Wang et al. 2019; Akopyan and Khashba 2017) explore the relations between the feature maps on both sides, which are actually abstract and hard to define.”