Risky Action Recognition in Lane Change Video Clips using Deep Spatiotemporal Networks with Segmentation Mask Transfer

By Ekim Yurtsever, Jacob Lambert, Chiyomi Miyajima, Eijiro Takeuchi, Kazuya Takeda, John H. L. Hansen. 2019

36 citations recorded by the source corpus.

Advanced driver assistance and automated driving systems rely on risk estimation modules to predict and avoid dangerous situations. Current methods use expensive sensor setups and complex processing pipelines, limiting their availability and robustness. To address these issues, we introduce a novel deep learning based driving risk assessment framework for classifying dangerous lane change behavior in short video clips captured by a monocular camera. First, semantic segmentation masks were generated from individual video frames with a pre-trained Mask R-CNN model. Then, frames overlayed with these masks were fed into a time distributed CNN-LSTM network with a final softmax classification layer. This network was trained on a semi-naturalistic lane change dataset with annotated risk labels. A comprehensive comparison of state-of-the-art pre-trained feature extractors was carried out to find the best network layout and training strategy. The best result, with a 0.937 AUC score, was obtained with the proposed framework. Our code and trained models are available open-source1.

Subfields: Artificial Intelligence, Automotive Engineering, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Kazuya Takeda
2. Jacob Lambert
3. Alexander Carballo
4. Ekim Yurtsever
5. Junichi Yamagishi
6. Tomoki Toda

In Automotive Engineering

1. Jacob Lambert
2. Alexander Carballo
3. Ekim Yurtsever
4. Kazuya Takeda
5. Ü. Özgüner

In Computer Vision and Pattern Recognition

1. Ekim Yurtsever
2. Alexander Carballo
3. Jacob Lambert
4. Kazuya Takeda
5. Ü. Özgüner

Citations in the corpus

• A Survey of Autonomous Driving: Common Practices and Emerging Technologies (2019, methodology): “We employed an open-source1 deep spatiotemporal video-based risk detection framework [197] to assess the image sequences shown in this figure.”
• Integrating Deep Reinforcement Learning with Model-based Path Planners for Automated Driving (2020, background): “The pipe usually continues with scene understanding [12], assessment [13], planning [14] and finally ends with motor control.”