MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learning
By Simon Vandenhende, Stamatios Georgoulis, Luc Van Gool. 2020
299 citations recorded by the source corpus.
In this paper, we argue about the importance of considering task interactions at multiple scales when distilling task information in a multi-task learning setup. In contrast to common belief, we show that tasks with high affinity at a certain scale are not guaranteed to retain this behaviour at other scales, and vice versa. We propose a novel architecture, namely MTI-Net, that builds upon this finding in three ways. First, it explicitly models task interactions at every scale via a multi-scale multi-modal distillation unit. Second, it propagates distilled task information from lower to higher scales via a feature propagation module. Third, it aggregates the refined task features from all scales via a feature aggregation unit to produce the final per-task predictions. Extensive experiments on two multi-task dense labeling datasets show that, unlike prior work, our multi-task model delivers on the full potential of multi-task learning, that is, smaller memory footprint, reduced number of calculations, and better performance w.r.t. single-task learning. The code is made publicly available: this https URL.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Michael Crawshaw
- Junjie Yan
- Wanli Ouyang
- Suman Saha
- Luc Van Gool
- Stamatios Georgoulis
- Xinzhu Ma
- David Bruggemann
In Computer Vision and Pattern Recognition
- Luc Van Gool
- Wanli Ouyang
- Xiaogang Wang
- Junjie Yan
- Ming-Hsuan Yang
- Dacheng Tao
- Andrea Vedaldi
- Chen Change Loy
Citations in the corpus
- Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference (2020, methodology): “On the other hand, decoder-focused approaches [63,65,66,62] utilize the multi-task feature representations learned at the encoding stage to distill cross-task information at the decoding stage, thus refining the original feature representations.”
- Geometry Uncertainty Projection Network for Monocular 3D Object Detection (2021, background): “Many works focus on task relation representation learning [28, 45, 48, 51, 52, 53].”
- Multi-Task Learning with Deep Neural Networks: A Survey (2020, methodology): “, 2019), and MTI-Net (Vandenhende et al., 2020) each take advantage of this phenomenon for the multi-task learning of computer vision tasks by making preliminary predictions for each task, then combining these predictions to produced final, refined outputs.”