Spatiotemporal Knowledge Distillation for Efficient Estimation of Aerial Video Saliency

By Jia Li, Kui Fu, Shengwei Zhao, Shiming Ge. 2019

30 citations recorded by the source corpus.

The performance of video saliency estimation techniques has achieved significant advances along with the rapid development of Convolutional Neural Networks (CNNs). However, devices like cameras and drones may have limited computational capability and storage space so that the direct deployment of complex deep saliency models becomes infeasible. To address this problem, this paper proposes a dynamic saliency estimation approach for aerial videos via spatiotemporal knowledge distillation. In this approach, five components are involved, including two teachers, two students and the desired spatiotemporal model. The knowledge of spatial and temporal saliency is first separately transferred from the two complex and redundant teachers to their simple and compact students, while the input scenes are also degraded from high-resolution to low-resolution to remove the probable data redundancy so as to greatly speed up the feature extraction process. After that, the desired spatiotemporal model is further trained by distilling and encoding the spatial and temporal saliency knowledge of two students into a unified network. In this manner, the inter-model redundancy can be removed for the effective estimation of dynamic saliency on aerial videos. Experimental results show that the proposed approach is comparable to 11 state-of-the-art models in estimating visual saliency on aerial videos, while its speed reaches up to 28,738 FPS and 1,490.5 FPS on the GPU and CPU platforms, respectively.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Jianping Gou
3. S. Maybank
4. Baosheng Yu
5. Baocai Yin
6. Hongchen Tan
7. Xiuping Liu
8. Xin Li

Citations in the corpus

• Knowledge Distillation: A Survey (2020, methodology): “…including the design of knowledge (Hinton et al. 2015; Romero et al. 2015) and the loss functions for matching features or distributions matching (Huang and Wang 2017; Passalis and Tefas 2018; Zagoruyko and Komodakis 2017; Mirzadeh et al. 2020; Li et al. 2020d; Heo et al. 2019b; Asif et al. 2020).”
• KT-GAN: Knowledge-Transfer Generative Adversarial Network for Text-to-Image Synthesis (2020, background): “Knowledge Distillation (KD) [15], [34] with neural networks was pioneered by Hinton [8], which is a transfer learning method that aims to improve the training of a student network by relying on knowledge borrowed from a powerful teacher network.”