Online detection of unusual events in videos via dynamic sparse coding

By Bin Zhao, Li Fei-Fei, Eric P. Xing. 2011

556 citations recorded by the source corpus.

Real-time unusual event detection in video stream has been a difficult challenge due to the lack of sufficient training information, volatility of the definitions for both normality and abnormality, time constraints, and statistical limitation of the fitness of any parametric models. We propose a fully unsupervised dynamic sparse coding approach for detecting unusual events in videos based on online sparse re-constructibility of query signals from an atomically learned event dictionary, which forms a sparse coding bases. Based on an intuition that usual events in a video are more likely to be reconstructible from an event dictionary, whereas unusual events are not, our algorithm employs a principled convex optimization formulation that allows both a sparse reconstruction code, and an online dictionary to be jointly inferred and updated. Our algorithm is completely un-supervised, making no prior assumptions of what unusual events may look like and the settings of the cameras. The fact that the bases dictionary is updated in an online fashion as the algorithm observes more data, avoids any issues with concept drift. Experimental results on hours of real world surveillance video and several Youtube videos show that the proposed algorithm could reliably locate the unusual events in the video sequence, outperforming the current state-of-the-art methods.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

C3D: Generic Features for Video Analysis (2014, background): “The computer vision community has been working on video analysis for decades and tackled different problems such as action recognition [2, 31, 40], abnormal event detection [3, 7, 55], and activity understanding [29].”
Abnormal Event Detection at 150 FPS in MATLAB (2013, methodology): “Second/Frame Platform CPU Memory [22] 2 MATLAB 7.”
Abnormal event detection in crowded scenes using sparse representation (2013, background): “Wrong Direction No-Pay Total False Alarm Ground truth [1] 21/9 10/- 31/9 -/Adam[1] 17/9 -/- 17/9 4/2 Ours 21/9 6/- 27/9 4/0 Ground truth [16] 26/9 13/3 39/12 -/Kim [16] 24/9 8/3 32/12 6/3 Zhao [38] 25/9 9/3 34/12 5/2”
Learning Spatiotemporal Features with 3D Convolutional Networks (2014, background): “s. It performs well on various video classiﬁcation problems such as object recognition, scene classiﬁ- cation, action similarity labeling, sport classiﬁcation, and action recognition. event detection [3,7,55], and activity understanding [29]. Considerable progress has been made in these individual p”
Detecting anomalous events in videos by learning deep representations of appearance and motion (2017, methodology): “As baseline methods we consider recent approaches adopting the same dataset, including Spatio-Temporal Composition (STC) [33], MPPCA [37], Spatio-Temporal Oriented Energy (STOE) [34], Dynamic Sparse Coding (DSC) [54], Sparse Reconstruction [4] and Local Optical Flow [49].”
Remembering history with convolutional LSTM for anomaly detection (2017, methodology): “Actually, training different models for different datasets are also commonly used in many existing works [16][5].”
MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection (2021, methodology): “[33, 16, 7, 17, 15, 5, 36] , which encodes the usual pattern with only normal training samples, and then detects the distinctive encoded patterns as anomalies.”
Abnormal Event Detection in Videos using Spatiotemporal Autoencoder (2017, background): “A popular approach adopted by researchers in this area is to ﬁrst learn the normal patterns from the training videos, then anomalies are detected as events deviated from the normal patterns [13, 2, 32, 12].”
Object-Centric Auto-Encoders and Dummy Anomalies for Abnormal Event Detection in Video (2018, background): “Abnormal event detection is commonly formalized as an outlier detection task [2, 5, 6, 9, 14, 15, 18, 23, 25, 26, 29, 36, 37, 38, 39], in which the main approach is to learn a model of familiarity from training videos and label the detected outliers as abnormal.”
Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection (2019, methodology): “Due to the critical application scenario, a series of methods are specifically designed for video anomaly detection [24, 44, 11, 21].”
Anomaly Detection in Video Sequence With Appearance-Motion Correspondence (2019, methodology): “Such sparsity-based methods have achieved state-of-the-art performances [6, 55].”
Learning Memory-Guided Normality for Anomaly Detection (2020, methodology): “Reconstructive models encode normal patterns using representation learning methods such as an AE [48, 36], a sparse dictionary learning [6, 49, 24], and a generative model [43].”
A Revisit of Sparse Coding Based Anomaly Detection in Stacked RNN Framework (2017, methodology): “To preserve the similarity between the neighboring frames, motivated by the work [29], we propose a Temporally-coherent Sparse Coding (TSC).”
An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos (2018, unknown): “Histogram of oriented optical ﬂow and oriented gradients [30] is a base-line used in anomaly detection and crowd analysis, [31], [32], [33].”
Graph Convolutional Label Noise Cleaner: Train a Plug-And-Play Action Classifier for Anomaly Detection (2019, background): “Sparse reconstruction [41, 31, 13, 67] is also another popular approach for usual pattern modeling.”