Future Frame Prediction for Anomaly Detection - A New Baseline
By Wen Liu, Weixin Luo, Dongze Lian, Shenghua Gao. 2017
1,395 citations recorded by the source corpus.
Anomaly detection in videos refers to the identification of events that do not conform to expected behavior. However, almost all existing methods tackle the problem by minimizing the reconstruction errors of training data, which cannot guarantee a larger reconstruction error for an abnormal event. In this paper, we propose to tackle the anomaly detection problem within a video prediction framework. To the best of our knowledge, this is the first work that leverages the difference between a predicted future frame and its ground truth to detect an abnormal event. To predict a future frame with higher quality for normal events, other than the commonly used appearance (spatial) constraints on intensity and gradient, we also introduce a motion (temporal) constraint in video prediction by enforcing the optical flow between predicted frames and ground truth frames to be consistent, and this is the first work that introduces a temporal constraint into the video prediction task. Such spatial and motion constraints facilitate the future frame prediction for normal events, and consequently facilitate to identify those abnormal events that do not conform the expectation. Extensive experiments on both a toy dataset and some publicly available datasets validate the effectiveness of our method in terms of robustness to the uncertainty in normal events and the sensitivity to abnormal events. All codes are released in https://github.com/StevenLiuWen/ano_pred_cvpr2018.
Subfields: Artificial Intelligence, Biomedical Engineering, Computer Vision and Pattern Recognition, Anomaly & Outlier Detection
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Kristen Grauman
- Trevor Darrell
- Luc Van Gool
- Abhinav Kumar Gupta
- Mohit Bansal
- Sergey Levine
- Cordelia Schmid
- Jitendra Malik
In Biomedical Engineering
- Kristen Grauman
- Rohit Girdhar
- Giovanni Maria Farinella
- Antonino Furnari
- Kris M. Kitani
- C. V. Jawahar
- Dima Damen
- Lorenzo Torresani
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Kristen Grauman
- Jitendra Malik
- Dacheng Tao
- Trevor Darrell
- Ming-Hsuan Yang
- Xiaogang Wang
- Cordelia Schmid
In Anomaly & Outlier Detection
- Jianfeng Feng
- Ling Shao
- Jean Meunier
- Radu Tudor Ionescu
- Fahad Shahbaz Khan
- Guansong Pang
- Trong-Nguyen Nguyen
- Weishi Zheng
Citations in the corpus
- Ego4D: Around the World in 3,000 Hours of Egocentric Video (2021, background): “Previous work considers future localization [113, 120, 174, 230], action anticipation [76, 77, 86, 118, 127, 219], next active object prediction [20, 74], future event prediction [149, 167], and future frame prediction [145, 146, 153, 215, 218, 227].”
- Object-Centric Auto-Encoders and Dummy Anomalies for Abnormal Event Detection in Video (2018, methodology): “Compared to most of the recent works [13, 21, 22, 24, 33], our method provides an absolute gain of more than 5% in terms of frame-level AUC.”
- A Review on Deep Learning Techniques for Video Prediction (2020, background): “[10], prediction of pedestrian trajectories in traffic [11], anomaly detection [12], precipitation nowcasting [13], [14], and autonomous driving [15].”
- Generalized Out-of-Distribution Detection: A Survey (2021, neutral): “DeepmodelsincludeAEs(Chen et al., 2018), VAEs (An & Cho, 2015), GANs (Zenati et al., 2018), and U-Net (Liu et al., 2018b).”
- Anomaly Detection via Reverse Distillation from One-Class Embedding (2022, background): “The wide applications of AD, such as industrial defect detection [3], medical out-of-distribution detection [50], and video surveillance [24], makes it a critical task as well as a spotlight.”
- Visual Memorability for Robotic Interestingness via Unsupervised Online Learning (2020, methodology): “he rst place at the DARPA SubT Tunnel Circuit. 10 Chen Wang et al. Table 2: The comparison with the state-of-the-art method on AUC-OP. (a) The SubT Normal Category. Methods = 1 = 2 = 3 baseline [25] 0.622 0.798 0.904 ours 0.662 0.842 0.923 (b) The SubT Dicult Category. Methods = 1 = 2 = 4 base”
- Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection (2019, methodology): “[24] train a frame prediction network by incorporating different techniques including gradient loss, optical flow, and adversarial training.”
- What comprises a good talking-head video generation?: A Survey and Benchmark (2020, neutral): “2019a), and video prediction (Liu et al. 2018; Minderer et al. 2019).”
- PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization (2020, neutral): “For further evaluation, we also test PaDiM on the Shanghai Tech Campus (STC) Dataset [8] that simulates video surveillance from a static camera.”