Self-Training Multi-Sequence Learning with Transformer for Weakly Supervised Video Anomaly Detection

By Sijie Li, Licheng Jiao. AAAI Conference on Artificial Intelligence, 2022

304 citations recorded by the source corpus.

Weakly supervised Video Anomaly Detection (VAD) using Multi-Instance Learning (MIL) is usually based on the fact that the anomaly score of an abnormal snippet is higher than that of a normal snippet. In the beginning of training, due to the limited accuracy of the model, it is easy to select the wrong abnormal snippet. In order to reduce the probability of selection errors, we first propose a Multi-Sequence Learning (MSL) method and a hinge-based MSL ranking loss that uses a sequence composed of multiple snippets as an optimization unit. We then design a Transformer-based MSL network to learn both video-level anomaly probability and snippet-level anomaly scores. In the inference stage, we propose to use the video-level anomaly probability to suppress the fluctuation of snippet-level anomaly scores. Finally, since VAD needs to predict the snippet-level anomaly scores, by gradually reducing the length of selected sequence, we propose a self-training strategy to gradually refine the anomaly scores. Experimental results show that our method achieves significant improvements on ShanghaiTech, UCF-Crime, and XD-Violence.

Subfields: Artificial Intelligence, Biomedical Engineering, Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/84f3290630be8dff726ca43195ed6c82bb65d0d9

Citations in the corpus