Learning Deep Representations of Appearance and Motion for Anomalous Event Detection

By Dan Xu, Elisa Ricci, Yan Yan, Jingkuan Song, Nicu Sebe. British Machine Vision Conference, 2015

541 citations recorded by the source corpus.

We present a novel unsupervised deep learning framework for anomalous event detection in complex video scenes. While most existing works merely use hand-crafted appearance and motion features, we propose Appearance and Motion DeepNet (AMDN) which utilizes deep neural networks to automatically learn feature representations. To exploit the complementary information of both appearance and motion patterns, we introduce a novel double fusion framework, combining both the benefits of traditional early fusion and late fusion strategies. Specifically, stacked denoising autoencoders are proposed to separately learn both appearance and motion features as well as a joint representation (early fusion). Based on the learned representations, multiple one-class SVM models are used to predict the anomaly scores of each input, which are then integrated with a late fusion strategy for final anomaly detection. We evaluate the proposed method on two publicly available video surveillance datasets, showing competitive performance with respect to state of the art approaches.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/60fef33549f57f5cbb6712a510c3a444ab682429

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

A Revisit of Sparse Coding Based Anomaly Detection in Stacked RNN Framework (2017, methodology): “Deep learning approaches have demonstrated its successes for image classification [15], object recognition [9], as well as anomaly detection [11][27].”
Generative Probabilistic Novelty Detection with Adversarial Autoencoders (2018, methodology): “[26, 27] used deep learning based autoencoders to learn the model of normal behaviors and employed a reconstruction loss to detect outliers.”
Classification-Reconstruction Learning for Open-Set Recognition (2018, background): “However, most of the recent anomaly-detection literature focuses on incorporating domain knowledge specific to the task at hand, such as cues from videos [47, 15], and they cannot be used to build a generic-purpose open-set classifiers.”
Deep Learning for Anomaly Detection (2020, methodology): “classification on the MNIST data [79]. Another research line in this category is to explicitly train a deep feature extraction model rather than a pre-trained model for the downstream anomaly scoring [45, 66, 164, 169]. Particularly, in [164], three separate autoencoder networks are trained to learn”
Unmasking the Abnormal Events in Video (2017, methodology): “Other approaches have employed deep features [24] or locality sensitive hashing filters [25] to achieve better results.”
Deep One-Class Classification (2018, methodology): “This allows for the use of autoencoders in mixed approaches (Xu et al., 2015; An-drews et al., 2016; Erfani et al., 2016; Sabokrou et al., 2016), by plugging the learned embeddings into classical AD meth-ods, but also in fully deep approaches, by directly employing the reconstruction error as an…”
Detecting anomalous events in videos by learning deep representations of appearance and motion (2017, background): “This paper extends our previous work in [24].”
Spatio-Temporal AutoEncoder for Video Anomaly Detection (2017, methodology): “…59.0 40.0 69.3 30.0 - - SF[16] 67.5 31.0 55.6 42.0 - - SF+MPPCA[14] 66.8 32.0 61.3 36.0 - - MDT[14] 81.8 25.0 82.9 25.0 - - SCL[12] 91.8 15.0 - - - - AMDN[32] 92.1 The comparison of our method with several state-of-the-art approaches on UCSD Pedestrian and CUHK Avenue datasets is shown in Table 2.”
Future Frame Prediction for Anomaly Detection - A New Baseline (2017, background): “In the work [36], Xu et al . de-sign a multi-layer auto-encoder for feature learning, which demonstrates the effectiveness of deep learning features.”
Remembering history with convolutional LSTM for anomaly detection (2017, background): “[16] propose a multi-layer auto-encoder for representation learning.”
Adversarially Learned One-Class Classifier for Novelty Detection (2018, methodology): “For data representation, low level features [4], high level features ( e.g ., trajectories [29]), deeply learned features [48, 37, 39] are used in the literature.”
Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes (2016, unknown): “Work reported in [5, 6] models normal events based on a set of representative features which are learned on auto-encoders [53].”
Real-World Anomaly Detection in Surveillance Videos (2018, background): “Recently, [18, 40] used deep learning based autoencoders to learn the model of normal behaviors and employed reconstruction loss to detect anomalies.”
Behavior-Driven Synthesis of Human Dynamics (2021, background): “Understanding human appearance, posture and behavior are key problems of computer vision with numerous applications in autonomous driving [39, 41, 23], surveillance [12, 50, 60], medical treatment [6, 54] and beyond.”
Learning Temporal Regularity in Video Sequences (2016, methodology): “[30] proposed a deep model for anomalous event detection that uses a stacked autoencoder for feature learning and a linear classifier for event classification.”