Attentive Single-Tasking of Multiple Tasks

By Kevis-Kokitsi Maninis, Ilija Radosavovic, Iasonas Kokkinos. 2019

292 citations recorded by the source corpus.

In this work we address task interference in universal networks by considering that a network is trained on multiple tasks, but performs one task at a time, an approach we refer to as ``single-tasking multiple tasks''. The network thus modifies its behaviour through task-dependent feature adaptation, or task attention. This gives the network the ability to accentuate the features that are adapted to a task, while shunning irrelevant ones. We further reduce task interference by forcing the task gradients to be statistically indistinguishable through adversarial training, ensuring that the common backbone architecture serving all tasks is not dominated by any of the task-specific gradients. Results in three multi-task dense labelling problems consistently show: (i) a large reduction in the number of parameters while preserving, or even improving performance and (ii) a smooth trade-off between computation and multi-task accuracy. We provide our system's code and pre-trained models at http://https://github.com/facebookresearch/astmt.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference (2020, methodology): “To suppress negative transfer, several approaches [6,21,59,16,69,56,39] dynamically re-weight each task’s loss function or re-order the task learning, to find a ‘sweet spot’ where individual task performance does not degrade significantly.”
Gradient Surgery for Multi-Task Learning (2020, background): “…learning problem based on multiple modules or paths (Fernando et al., 2017; Devin et al., 2016; Misra et al., 2016b; Rusu et al., 2016b; Rosenbaum et al., 2018; Vandenhende et al., 2019; Rosenbaum et al., 2018), or using attention-based architectures (Liu et al., 2018; Maninis et al., 2019).”
Which Tasks Should Be Learned Together in Multi-task Learning? (2019, background): “, 2019) adds a per-task attention layer after each layer, and (Maninis et al., 2019) adds task specific residual paths and adversarial training to match gradients.”
Surgical data science – from concepts toward clinical translation (2020, background): “A possible solution to this problem might lie in the approach of attentive single-tasking (Maninis et al., 2019).”
Omnivore: A Single Model for Many Visual Modalities (2022, background): “Our work is also related to studies on multi-task learning [14], which develop models that output predictions for multiple tasks on the same input [23, 27, 44, 58, 61, 102].”
Side-Tuning: Network Adaptation via Additive Side Networks (2019, unknown): “In Figure 9, we show all the losses normalized by single task loss (independent) as presented in [2].”
Branched Multi-Task Networks: Deciding what layers to share (2019, background): “Finally, multi-task networks may outperform their single-task counterparts (Kendall et al., 2018; Xu et al., 2018; Sener & Koltun, 2018; Maninis et al., 2019).”
MTL-NAS: Task-Agnostic Neural Architecture Search Towards General-Purpose Multi-Task Learning (2020, methodology): “Our work leverages NAS to move towards GP-MTL, which (or whose search space) is compatible with any task combinations [27,29,36–38,40,42,52,67].”
Factorizing Knowledge in Neural Networks (2022, background): “Some recent solutions explore on the decomposition between shared and task-speciﬁc processing [40,25,69].”
Federated Adversarial Debiasing for Fair and Transferable Representations (2021, background): “Adversarial Learning has been widely applied in various domains, such as neural language recognition [27], image-to-image (dense) prediction [31], image generation [12], and etc. Conceptually, adversarial learning aims to solve a two-player (or multi-player) game between two adversarial objectives,…”
AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning (2019, methodology): “MTAN [33] and Attentive Single-Tasking [37], introduce a task-specific attention branch per task paired with the shared backbone.”
When Does Self-supervision Improve Few-shot Learning? (2019, background): “However, there is significant evidence that training on multiple tasks together often hurts performance on individual tasks [30,37].”
MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learning (2020, methodology): “Arguably, this might be the reason for the only moderate performance improvements achieved by this group of works (see [29]).”