Real-World Anomaly Detection in Surveillance Videos
By Waqas Sultani, Mubarak Shah. 2018
2,070 citations recorded by the source corpus.
Surveillance videos are able to capture a variety of realistic anomalies. In this paper, we propose to learn anomalies by exploiting both normal and anomalous videos. To avoid annotating the anomalous segments or clips in training videos, which is very time consuming, we propose to learn anomaly through the deep multiple instance ranking framework by leveraging weakly labeled training videos, i.e. the training labels (anomalous or normal) are at video-level instead of clip-level. In our approach, we consider normal and anomalous videos as bags and video segments as instances in multiple instance learning (MIL), and automatically learn a deep anomaly ranking model that predicts high anomaly scores for anomalous video segments. Furthermore, we introduce sparsity and temporal smoothness constraints in the ranking loss function to better localize anomaly during training. We also introduce a new large-scale first of its kind dataset of 128 hours of videos. It consists of 1900 long and untrimmed real-world surveillance videos, with 13 realistic anomalies such as fighting, road accident, burglary, robbery, etc. as well as normal activities. This dataset can be used for two tasks. First, general anomaly detection considering all anomalies in one group and all normal activities in another group. Second, for recognizing each of 13 anomalous activities. Our experimental results show that our MIL method for anomaly detection achieves significant improvement on anomaly detection performance as compared to the state-of-the-art approaches. We provide the results of several recent deep learning baselines on anomalous activity recognition. The low recognition performance of these baselines reveals that our dataset is very challenging and opens more opportunities for future work. The dataset is available at: http://crcv.ucf.edu/projects/real-world/
Subfields: Artificial Intelligence, Biomedical Engineering, Computer Networks and Communications, Computer Vision and Pattern Recognition, Anomaly & Outlier Detection
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Chunhua Shen
- Fahad Shahbaz Khan
- Mubarak Shah
- Ling Shao
- Luc Van Gool
- Silvio Savarese
- Ian David Reid
- Anton van den Hengel
In Biomedical Engineering
- Licheng Jiao
- Sijie Li
- Weishi Zheng
- Yingli Tian
- Xiaodong Yang
- Mubarak Shah
- Silvio Savarese
- Chunhua Shen
In Computer Networks and Communications
- Licheng Jiao
- Sijie Li
- Guansong Pang
- Chunhua Shen
- Radu Tudor Ionescu
- Fahad Shahbaz Khan
- Mariana-Iuliana Georgescu
- Jianfeng Feng
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Ming-Hsuan Yang
- Xiaogang Wang
- Andrew Zisserman
- Jitendra Malik
- Trevor Darrell
- Chunhua Shen
In Anomaly & Outlier Detection
- Licheng Jiao
- Sijie Li
- Radu Tudor Ionescu
- Fahad Shahbaz Khan
- Guansong Pang
- Mariana-Iuliana Georgescu
- Jianfeng Feng
- Weishi Zheng
Citations in the corpus
- End-to-End Learning of Visual Representations From Uncurated Instructional Videos (2019, neutral): “Multiple instance learning methods have been employed in many weakly-supervised video understanding problems including: person recognition in movies using scripts [12, 50, 61], anomaly detection [70], weakly supervised action classification [47, 68] and localization [17, 22, 82], co-reference resolu”
- Self-Supervised Spatiotemporal Feature Learning via Video Rotation Prediction. (2018, background): “With more videos flourishing on the internet, recognizing human actions from videos [37, 50, 9, 21, 41, 51, 52] has drawn increasing attention in computer vision community.”
- Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning (2021, methodology): “Moreover, our method unifies the representation learning and anomaly score learning by an `2-norm-based temporal feature ranking loss, enabling better separation between normal and abnormal feature representations, improving the exploration of weak labels compared to previous MIL methods [56, 62, 66”
- A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction (2021, methodology): “To show this, we conduct experiment on UCF-Crime dataset [7].”
- MLVU: A Comprehensive Benchmark for Multi-Task Long Video Understanding (2024, positive): “We leverage the surveillance video clips from UCF Crime dataset [37] for this task.”
- Anomaly Detection in Video via Self-Supervised and Multi-Task Learning (2020, methodology): “[48] and continued by others [65] less related to our study.”
- Self-Training Multi-Sequence Learning with Transformer for Weakly Supervised Video Anomaly Detection (2022, methodology): “We conduct sufficient experiments on the ShanghaiTech, UCF-Crime, and XD-Violence datasets.”
- Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision (2020, methodology): “Recently, several research [36,51] focus on weakly supervised violence detection, where only video-level labels are available in the training set.”
- CutPaste: Self-Supervised Learning for Anomaly Detection and Localization (2021, background): “Many problems from different vision applications are anomaly detection, including manufacturing defect detection [9, 5], medical image analysis [50, 48], and video surveillance [2, 31, 53].”
- Anomaly Detection in Video Sequence With Appearance-Motion Correspondence (2019, background): “[45]) or particular relevant attributes that can be used effectively for anomaly detection, in which the most common one is motion trajectory.”
- Deep Learning for Anomaly Detection (2020, methodology): “3 MIL [145] Keras https://git.”
- Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks (2019, background): “Intelligent tracking and surveillance systems used for city planning must be able to understand how crowds will interact to better manage infrastructure [5–8].”
- Behavior-Driven Synthesis of Human Dynamics (2021, background): “Understanding human appearance, posture and behavior are key problems of computer vision with numerous applications in autonomous driving [39, 41, 23], surveillance [12, 50, 60], medical treatment [6, 54] and beyond.”
- Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities (2022, background): “Anomaly detection in surveillance videos [48] and skill assessment [7, 13, 30, 53] are active research areas, but to the best of our knowledge, detecting mistakes in procedural activities has not been previously studied.”
- MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection (2021, methodology): “Some works focus on detecting anomalies in an offline manner [26, 29] or a coarse-grained manner [23, 32, 38, 26, 29], which do not meet the real-time monitoring requirements for real-world applications.”
- Graph Convolutional Label Noise Cleaner: Train a Plug-And-Play Action Classifier for Anomaly Detection (2019, methodology): “Following previous works [43, 20, 58], we plot the frame-level receiver operating characteristics (ROC) curve and compute an area under the curve (AUC) as the evaluation metric.”
- Object-Centric Auto-Encoders and Dummy Anomalies for Abnormal Event Detection in Video (2018, methodology): “Abnormal event detection in video has drawn a lot of attention in the past couple of years [7, 11, 12, 13, 14, 21, 22, 24, 27, 28, 31, 33, 34, 36, 37, 38], perhaps because it is considered a challenging task due to the commonly accepted definition of abnormal events, which relies on context.”