Abnormal event detection in videos using generative adversarial nets

By Mahdyar Ravanbakhsh, Moin Nabi, Enver Sangineto, Lucio Marcenaro, Carlo S. Regazzoni, Nicu Sebe. International Conference on Information Photonics, 2017

482 citations recorded by the source corpus.

In this paper we address the abnormality detection problem in crowded scenes. We propose to use Generative Adversarial Nets (GANs), which are trained using normal frames and corresponding optical-flow images in order to learn an internal representation of the scene normality. Since our GANs are trained with only normal data, they are not able to generate abnormal events. At testing time the real data are compared with both the appearance and the motion representations reconstructed by our GANs and abnormal areas are detected by computing local differences. Experimental results on challenging abnormality detection datasets show the superiority of the proposed method compared to the state of the art in both frame-level and pixel-level abnormality detection tasks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/9d5290fadb7625862a966e0330bd0f9e111fc99d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Generative Probabilistic Novelty Detection with Adversarial Autoencoders (2018, methodology): “The most recent approaches are based on learning deep network architectures [10, 11], and they tend to either learn a one-class classiﬁer [12, 11], or to somehow leverage as novelty score, the reconstruction error of the encoder-decoder architecture they are based on [13, 7].”
Adversarially Learned One-Class Classifier for Novelty Detection (2018, background): “In a concurrent work, [33] proposed to learn the generator as a reconstructor of normal events, and hence if it cannot properly reconstruct a chunk of the input frames, that chunk is considered an anomaly.”
Vision-based human activity recognition: a survey (2020, methodology): “The most commonly used and efficient approaches are: Auto-encoder [187, 221], Variational Autoencoders (VAE) [41] and Generative Adversarial Networks (GAN) [146].”
Learning Memory-Guided Normality for Anomaly Detection (2020, methodology): “, a convolutional AE (Conv-AE) [9], a 3D Conv-AE [50], a recurrent neural network (RNN) [29, 26, 25], and a generative adversarial network (GAN) [33].”
Anomaly Detection in Video via Self-Supervised and Multi-Task Learning (2020, methodology): “While some frameworks, such as [27, 33, 40, 41, 47], approach the problem from a global (frame-level) perspective, methods such as [7, 11, 21, 19, 28, 29, 31, 32, 44, 46, 64] extract features at a local level, e.”
An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos (2018, methodology): “Here we shall review the work done in [73] applied to anomaly detection in videos.”
A Review on Deep Learning Techniques for Video Prediction (2020, unknown): “[101] 2019 [45], [192], [193] cycleGAN [115], [119], [120], [194], [195] RGB RGB X `1, LoG,AL R ? ? ? 7 7”
Deep Learning for Anomaly Detection: A Survey (2019, background): “…several Generative Adversarial Networks-based Anomaly Detection (GAN-AD) frameworks (Li et al. [2018], Deecke et al. [2018], Schlegl et al. [2017], Ravanbakhsh et al. [2017b], Eide [2018]) proposed are shown to be effective in identifying anomalies on high dimensional and complex datasets.”
Object-Centric Auto-Encoders and Dummy Anomalies for Abnormal Event Detection in Video (2018, methodology): “Another problem with Ped1 is that some recent works report results only on a subset of 16 videos [27, 28, 36], while others [13, 25, 21, 22] report results on all 36 test videos.”
Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection (2021, methodology): “Anomaly detection methods can be classified into: dictionary learning methods [7–9, 14, 40, 55], change detection frameworks [11, 26, 38, 48], probability-based methods [1, 2, 16, 23, 29, 44, 45, 56, 61, 74], distance-based models [6, 10, 25, 27, 50, 51,53,57,58,63,65,68,70], and reconstruction-base”
AnomalyNet: An Anomaly Detection Network for Video Surveillance (2019, methodology): “Therefore, we just follow the common experiment setting in [34], [39], and [67] to conduct experiments on three individual scenes and report the average frame-level AUC and EER score for all the three scenarios in Table VI.”
Anomaly Detection in Video Sequence With Appearance-Motion Correspondence (2019, methodology): “Typical structures of image reconstruction and translation are usually employed and the difference between their output and ground truth is used to indicate the frame-level score [11, 37, 25].”
Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning (2021, background): “Although one-class classifiers (OCCs, also called unsupervised anomaly detection) trained exclusively with normal videos have been explored in this context [15,17,27,30,46, 47, 76], the best performing approaches explore a weaklysupervised setup using training samples with video-level label annotati”
Future Frame Prediction for Anomaly Detection - A New Baseline (2017, methodology): “In this section, we compare our method with different hand-craft features based method [11][18][25][34] and latest deep learning based methods [14][15][16][24][29][33].”