Multi-Task Learning as Multi-Objective Optimization
By Ozan Sener, Vladlen Koltun. 2018
1,788 citations recorded by the source corpus.
In multi-task learning, multiple tasks are solved jointly, sharing inductive bias between them. Multi-task learning is inherently a multi-objective problem because different tasks may conflict, necessitating a trade-off. A common compromise is to optimize a proxy objective that minimizes a weighted linear combination of per-task losses. However, this workaround is only valid when the tasks do not compete, which is rarely the case. In this paper, we explicitly cast multi-task learning as multi-objective optimization, with the overall objective of finding a Pareto optimal solution. To this end, we use algorithms developed in the gradient-based multi-objective optimization literature. These algorithms are not directly applicable to large-scale learning problems since they scale poorly with the dimensionality of the gradients and the number of tasks. We therefore propose an upper bound for the multi-objective loss and show that it can be optimized efficiently. We further prove that optimizing this upper bound yields a Pareto optimal solution under realistic assumptions. We apply our method to a variety of multi-task deep learning problems including digit classification, scene understanding (joint semantic segmentation, instance segmentation, and depth estimation), and multi-label classification. Our method produces higher-performing models than recent multi-task learning formulations or per-task training.
Subfields: Artificial Intelligence, Computational Mechanics, Computer Vision and Pattern Recognition, Optimization & Training, Meta & Continual Learning
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sergey Levine
- Yoshua Bengio
- Quoc V. Le
- Mohit Bansal
- Jianfeng Gao
- Dacheng Tao
- Pieter Abbeel
- Chelsea Finn
In Computational Mechanics
- Luc Van Gool
- Qiang Yang
- Iasonas Kokkinos
- Kevis-Kokitsi Maninis
- Simon Vandenhende
- Michael Crawshaw
- Stamatios Georgoulis
- Yu Zhang
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Dacheng Tao
- Trevor Darrell
- Xiaogang Wang
- Jitendra Malik
- Ming-Hsuan Yang
- Yi Yang
- Andrew Zisserman
In Optimization & Training
- Michael Crawshaw
- Sergey Levine
- Chelsea Finn
- Vladlen Koltun
- Tianbao Yang
- Zhe Zhao
- Tianhe Yu
- Abhishek Gupta
In Meta & Continual Learning
- Chelsea Finn
- Sergey Levine
- Tianhe Yu
- Michael Crawshaw
- Abhishek Gupta
- Luc Van Gool
- Maliheh Izadi
- Roberta Gismondi
Citations in the corpus
- A Survey on Multi-Task Learning (2017, background): “Moreover, the GradNorm [115] is devised to normalize gradients to balance the learning of multiple tasks and [116] proposes the gradient surgery to avoid the interference between task gradients.”
- Exploring Example Influence in Continual Learning (2022, positive): “Specifically, according to the KKT conditions [15], we have Referring to the study from Sener et al. [34], the optimal γ ∗ is easily computed as Thus, the SP Pareto influence of the training batch can be computed by This process can be seen in Fig.”
- Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference (2020, methodology): “Several works proposed to dynamically re-weight the loss function of each task [6,21,59,56], sort the order of task learning [16], or adapt the feature sharing between ‘related’ and ‘unrelated’ tasks [69], to mitigate the effect of negative transfer.”
- Gradient Surgery for Multi-Task Learning (2020, methodology): “…solved by simply applying a standard single-task algorithm with a suitable task identifier provided to the model, or a simple multi-head or multi-output model, a number of prior works (Parisotto et al., 2015; Rusu et al., 2016a; Sener & Koltun, 2018) have found this learning problem to be diffi-cult.”
- Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer (2019, methodology): “multi-objective optimization [12] leads to improved results compared to a naive mixing strategy.”
- Learning to Fuse Things and Stuff (2018, background): “[33] formulate multi-task learning as a multi-objective optimization problem.”
- Vision Transformers for Dense Prediction (2021, methodology): “We use multi-objective optimization [34] together with Adam [20] and set a learning rate of 1e−5 for the backbone and 1e−4 for the decoder weights.”
- MSeg: A Composite Dataset for Multi-Domain Semantic Segmentation (2020, methodology): “We compare MGDA and CCSA with our simple strategy of evenly mixing data in Table 5.”
- Quantum Machine Learning for 6G Communication Networks: State-of-the-Art and Vision for the Future (2019, neutral): “, [203].”
- Agree to Disagree: Adaptive Ensemble Knowledge Distillation in Gradient Space (2020, methodology): “[26] demonstrates that the obtained gradient direction will accommodate all teachers, and no tolerance of disagreement is allowed.”
- Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models (2020, methodology): “…learning task relatedness (Zhang & Yeung, 2012), routing task-specifc paths (Rusu et al., 2016; Rosenbaum et al., 2019), altering gradients directly (Kendall et al., 2018; Chen et al., 2018a; Du et al., 2018; Yu et al., 2020), or searching pareto solutions (Sener & Koltun, 2018; Lin et al., 2019).”
- Multi-Task Learning for Dense Prediction Tasks: A Survey (2020, methodology): “[22], [23] Other (Sec.”
- A Survey on Negative Transfer (2020, neutral): “Various techniques have been explored to remedy negative interference, such as altering the gradients directly [97], [98], weighting tasks [99], learning task relatedness [100], [101], routing networks [102], [103], and searching for Pareto solutions [104], [105], etc.”
- The operational medium-range deterministic weather forecasting can be extended beyond a 10-day lead time (2023, neutral): “Previousstudiesinthemulti-tasklearning 25,26 paradigmshowthat assigning different weights between different tasks is helpful for learning representations.”
- FairMOT: On the Fairness of Detection and Re-identification in Multiple Object Tracking (2020, methodology): “Multi-task Learning There is a large body of literature (Liu et al. 2019; Kendall et al. 2018; Chen et al. 2018b; Guo et al. 2018; Sener and Koltun 2018) on multi-task learning which may be used to balance the object detection and reID feature extraction tasks.”
- How to train your robot with deep reinforcement learning: lessons we have learned (2021, background): “…multiple tasks at once can present a challenge even for supervised learning problems due to different tasks being learned at different rates (Chen et al. 2018; Schaul et al. 2019) or the challenges in determining how to resolve conflicting gradient signals between tasks (Sener and Koltun 2018).”
- Controlled Text Generation as Continuous Optimization with Multiple Constraints (2021, background): “This phenomenon has also been studied in multi-task learning [36, 37, 57] where linear combination of multiple task losses is the most common approach and approaches for multi-objective gradient descent have been proposed.”
- Attentive Single-Tasking of Multiple Tasks (2019, methodology): “Following a probabilistic treatment [25] re-weigh the losses according to each task’s uncertainty, while Sener and Koltun [56] estimate an adaptive weighting of the different task losses based on a pareto-optimal formulation of MTL. Similarly, [20] provide a MTL framework where tasks are…”