Rendering Physically Correct Raindrops on Windshields for Robustness Verification of Camera-based Object Recognition

By Alexander von Bernuth, Georg Volk, Oliver Bringmann. 2018

23 citations recorded by the source corpus.

Recent developments in the field of autonomous cars indicate the appearance of those vehicles on the streets of every city in the near future. This urban driving requires zero error tolerance. In order to guarantee safety requirements self-driving cars and the used software have to pass exhaustive tests under as many different conditions as possible. The more versatile the considered influences and the more thorough the tests made under those influences, the safer the car will drive under real conditions. Unfortunately, it is very time and resource intensive to record the same test set of images over and over again, every time producing, or hoping for, specific conditions; especially when using real test vehicles. This is where environment simulation comes into play. This research investigates the simulation of environmental influences which may affect the sensors used in autonomous vehicles, in particular how raindrops resting on a windshield affect cameras as they may occlude large parts of the field of view. We propose a novel method to render these raindrops using Continuous Nearest Neighbor search leveraging the benefits of R-trees. The 3D scene in front of the camera, which is generated from stereo images, reflects physically correct in these drops. This leads to near photo-realistic simulated results. The derived images may be used to extend the training data sets used for machine learning without being forced to capture new real pictures.

Subfields: Automotive Engineering, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Automotive Engineering

Matthias Bethge

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming (2019, background): “…many speciﬁc approaches like modelling weather conditions [Sakaridis et al., 2018a,b, Volk et al., 2019, Bernuth et al., 2019, Hospach et al., 2016, Bernuth et al., 2018] or collecting real [Wen et al., 2015, Yu et al., 2018, Che et al., 2019, Caesar et al., 2019] and artiﬁcial [Gaidon et al.,…”