A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction

By Zhian Liu, Yongwei Nie, Chengjiang Long, Qing Zhang, Guiqing Li. IEEE International Conference on Computer Vision, 2021

359 citations recorded by the source corpus.

In this paper, we propose HF2-VAD, a Hybrid framework that integrates Flow reconstruction and Frame prediction seamlessly to handle Video Anomaly Detection. Firstly, we design the network of ML-MemAE-SC (Multi-Level Memory modules in an Autoencoder with Skip Connections) to memorize normal patterns for optical flow reconstruction so that abnormal events can be sensitively identified with larger flow reconstruction errors. More importantly, conditioned on the reconstructed flows, we then employ a Conditional Variational Autoencoder (CVAE), which captures the high correlation between video frame and optical flow, to predict the next frame given several previous frames. By CVAE, the quality of flow reconstruction essentially influences that of frame prediction. Therefore, poorly reconstructed optical flows of abnormal events further deteriorate the quality of the final predicted future frame, making the anomalies more detectable. Experimental results demonstrate the effectiveness of the proposed method. Code is available at https://github.com/LiUzHiAn/hf2vad.

Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/5cccc8bab11927e5527dd5b917ed4c306a2ccf49

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Fahad Shahbaz Khan
2. Radu Tudor Ionescu
3. Mubarak Shah
4. Thomas Baltzer Moeslund
5. Luc Van Gool
6. Chen Sun
7. Andrew Zisserman
8. Cordelia Schmid

In Biomedical Engineering

1. Radu Tudor Ionescu
2. Fahad Shahbaz Khan
3. Mubarak Shah

In Computer Vision and Pattern Recognition

1. Fahad Shahbaz Khan
2. Luc Van Gool
3. Radu Tudor Ionescu
4. Ming-Hsuan Yang
5. Mubarak Shah
6. Dacheng Tao
7. Philip H. S. Torr
8. Thomas Baltzer Moeslund

Citations in the corpus

• Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection (2021, methodology): “For the methods [18,34,37,39,49,79] chosen to serve as underlying models for SSPCAB, we use the official code from the repositories provided by the corresponding authors, inheriting the hyperparameters, e.”