Anomaly Detection in Video via Self-Supervised and Multi-Task Learning

By Mariana-Iuliana Georgescu, Antonio Barbalau, Radu Tudor Ionescu, Fahad Shahbaz Khan, Marius Claudiu Popescu, Mubarak Shah. Computer Vision and Pattern Recognition, 2020

355 citations recorded by the source corpus.

Anomaly detection in video is a challenging computer vision problem. Due to the lack of anomalous events at training time, anomaly detection requires the design of learning methods without full supervision. In this paper, we approach anomalous event detection in video through self-supervised and multi-task learning at the object level. We first utilize a pre-trained detector to detect objects. Then, we train a 3D convolutional neural network to produce discriminative anomaly-specific information by jointly learning multiple proxy tasks: three self-supervised and one based on knowledge distillation. The self-supervised tasks are: (i) discrimination of forward/backward moving objects (arrow of time), (ii) discrimination of objects in consecutive/intermittent frames (motion irregularity) and (iii) reconstruction of object-specific appearance information. The knowledge distillation task takes into account both classification and detection information, generating large prediction discrepancies between teacher and student models when anomalies occur. To the best of our knowledge, we are the first to approach anomalous event detection in video as a multi-task learning problem, integrating multiple self-supervised and knowledge distillation proxy tasks in a single architecture. Our lightweight architecture outperforms the state-of-the-art methods on three benchmarks: Avenue, ShanghaiTech and UCSD Ped2. Additionally, we perform an ablation study demonstrating the importance of integrating self-supervised learning and normality-specific distillation in a multi-task learning setting.

Subfields: Artificial Intelligence, Biomedical Engineering, Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/57b2198f9a8df773425aa6cc88c9870cb07779e2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Biomedical Engineering

In Computer Networks and Communications

Citations in the corpus

Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning (2021, methodology): “Some recent studies [12, 45] recommend using the regionbased detection criterion (RBDC) and the track-based detection criterion (TBDC) to complement the AUC measure, but these two measures are inapplicable in the weaklysupervised setting.”
Self-Supervised Representation Learning: Introduction, advances, and challenges (2021, neutral): “Across these modalities, it can also be applied to boost diverse downstream tasks, including not only simple recognition but also detection and localization [16], dense prediction (signal transformation) [16], anomaly detection [17], and so on.”
Generalized Out-of-Distribution Detection: A Survey (2021, neutral): “Non-parametric density estimation handles more complex scenarios (Izenman, 1991) with histograms (Van Ryzin, 1973; Xie et al., 2012; Kind et al., 2009; Goldstein & Dengel, 2012) and kernel density estimation (KDE) (Parzen, 1962; Desforges et al., 1998; Hu et al., 2018).”
Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection (2021, background): “superpixels [36], future frames [37], middle bounding boxes [17], among others.”