Fast Scene Understanding for Autonomous Driving

By Davy Neven, Bert De Brabandere, Stamatios Georgoulis, Marc Proesmans, Luc Van Gool. 2017

77 citations recorded by the source corpus.

Most approaches for instance-aware semantic labeling traditionally focus on accuracy. Other aspects like runtime and memory footprint are arguably as important for real-time applications such as autonomous driving. Motivated by this observation and inspired by recent works that tackle multiple tasks with a single integrated architecture, in this paper we present a real-time efficient implementation based on ENet that solves three autonomous driving related tasks at once: semantic scene segmentation, instance segmentation and monocular depth estimation. Our approach builds upon a branched ENet architecture with a shared encoder but different decoder branches for each of the three tasks. The presented method can run at 21 fps at a resolution of 1024x512 on the Cityscapes dataset without sacrificing accuracy compared to running each task separately.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference (2020, background): “On the one hand, encoder-focused approaches [41,25,36,10,43,33,1,61] emphasize learning feature representations from multi-task supervisory signals by employing architectures that encode shared and task-speciﬁc information.”
Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation (2019, methodology): “Finally, our model employs the parallel multi-head prediction framework similar to [68, 36, 55].”
Branched Multi-Task Networks: Deciding what layers to share (2019, methodology): “Second, as features in the shared layers do not need to be calculated repeatedly for the different tasks, the overall inference speed is often higher [31, 34].”
NeurAll: Towards a Unified Visual Perception Model for Automated Driving (2019, background): “[29] share similar design but for fewer tasks without multi-stream design explained later.”
Towards End-to-End Lane Detection: an Instance Segmentation Approach (2018, background): “To increase performance, both in terms of speed and accuracy [27], these two parts are jointly trained in a multi-task network (see Fig.”
Towards Bounding-Box Free Panoptic Segmentation (2020, methodology): “To achieve this we exploit previous works in non-proposal based methods for instance segmentation [3, 5, 26].”
AuxNet: Auxiliary tasks enhanced Semantic Segmentation for Automated Driving (2019, background): “Multi-task learning (Kokkinos, 2017), (Chen et al., 2018b), (Neven et al., 2017) has been gaining significant popularity over the past few years as it has proven to be very efficient for embedded deployment.”
Box2Pix: Single-Shot Instance Segmentation by Assigning Pixels to Object Boxes (2018, background): “Many works use loss variants of typical metric learning problems in order to address instance segmentation by predicting embedding vectors that differ across objects [9], [10], [24], [25].”
Rethinking Self-driving: Multi-task Knowledge for Better Generalization and Accident Explanation Ability (2018, background): “Branched E-Net(Neven et al., 2017) not only infers for segmentation map, but also depth map of current driving scenarios.”
MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learning (2020, background): “r et al. [49] modeled the structure of the visual tasks’ space by nding transfer learning dependencies across a dictionary of twenty six tasks. Despite the progress reported by these or similar works [39,27,32,25,44], the joint learning of multiple tasks can lead to single-task performance degradati”