Auxiliary Tasks in Multi-task Learning

By Lukas Liebel, Marco Körner. 2018

314 citations recorded by the source corpus.

Multi-task convolutional neural networks (CNNs) have shown impressive results for certain combinations of tasks, such as single-image depth estimation (SIDE) and semantic segmentation. This is achieved by pushing the network towards learning a robust representation that generalizes well to different atomic tasks. We extend this concept by adding auxiliary tasks, which are of minor relevance for the application, to the set of learned tasks. As a kind of additional regularization, they are expected to boost the performance of the ultimately desired main tasks. To study the proposed approach, we picked vision-based road scene understanding (RSU) as an exemplary application. Since multi-task learning requires specialized datasets, particularly when using extensive sets of tasks, we provide a multi-modal dataset for multi-task RSU, called synMT. More than 2.5 $\cdot$ 10^5 synthetic images, annotated with 21 different labels, were acquired from the video game Grand Theft Auto V (GTA V). Our proposed deep multi-task CNN architecture was trained on various combination of tasks using synMT. The experiments confirmed that auxiliary tasks can indeed boost network performance, both in terms of final results and training time.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Stratospheric Aerosol Injection as a Deep Reinforcement Learning Problem (2019, background): “Preliminary simulation results suggest that emulator training would likely benefit from auxiliary tasks (Liebel, 2018) related to cloud cover prediction (see Figure 1).”
NeurAll: Towards a Unified Visual Perception Model for Automated Driving (2019, methodology): “[41] performed semantic segmentation using auxiliary tasks like time, weather, etc.”
Learning to Predict User-Defined Types (2023, background): “New approaches include dynamically weighing tasks from gradients [57] and uncertainty [32], [33].”
NeurAll: Towards a Unified Model for Visual Perception in Automated Driving (2019, methodology): “Liebel et al. [40] performed semantic segmentation using auxiliary tasks like time, weather, etc.”
AuxNet: Auxiliary tasks enhanced Semantic Segmentation for Automated Driving (2019, methodology): “In (Liebel and Körner, 2018) semantic segmentation is performed using auxiliary tasks like time, weather, etc.”
Multimodal Motion Prediction with Stacked Transformers (2021, methodology): “Since the total loss function can be viewed as the summation of multiple distinct tasks, we use multi-task learning approach in [18, 23] to balance them. where σ i , i ∈ { 1 , 2 , 3 } are learnable loss weights.”