Stochastic Filter Groups for Multi-Task CNNs: Learning Specialist and Generalist Convolution Kernels

By Felix J. S. Bragman, Ryutaro Tanno, Sébastien Ourselin, Daniel C. Alexander, M. Jorge Cardoso. 2019

97 citations recorded by the source corpus.

The performance of multi-task learning in Convolutional Neural Networks (CNNs) hinges on the design of feature sharing between tasks within the architecture. The number of possible sharing patterns are combinatorial in the depth of the network and the number of tasks, and thus hand-crafting an architecture, purely based on the human intuitions of task relationships can be time-consuming and suboptimal. In this paper, we present a probabilistic approach to learning task-specific and shared representations in CNNs for multi-task learning. Specifically, we propose "stochastic filter groups" (SFG), a mechanism to assign convolution kernels in each layer to "specialist" and "generalist" groups, which are specific to and shared across different tasks, respectively. The SFG modules determine the connectivity between layers and the structures of task-specific and shared representations in the network. We employ variational inference to learn the posterior distribution over the possible grouping of kernels and network parameters. Experiments demonstrate the proposed method generalises across multiple tasks and shows improved performance over baseline methods.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference (2020, background): “On the one hand, encoder-focused approaches [41,25,36,10,43,33,1,61] emphasize learning feature representations from multi-task supervisory signals by employing architectures that encode shared and task-specific information.”
Branched Multi-Task Networks: Deciding what layers to share (2019, methodology): “To alleviate the heavy computation burden, a recent work [23] implemented an evolutionary architecture search for multi-task networks, while other researchers explored more viable alternatives, like routing [39], stochastic filter grouping [5], and feature partitioning [35].”
Multi-Task Learning with Deep Neural Networks: A Survey (2020, background): “(Bragman et al., 2019) proposes Stochastic Filter Groups (SFGs), in which the assignment of a convolutional filter to task-specific or shared is learned through variational inference.”
Multi-Task Learning for Dense Prediction Tasks: A Survey (2020, methodology): “[97] implemented an evolutionary architecture search for MTL, while others explored alternatives like branched MTL networks [9], [10], [11], [12], routing [65], stochastic filter grouping [56], and feature partitioning [68].”
AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning (2019, methodology): “Only a few methods have attempted to learn multibranch network architectures, using greedy optimization based on task affinity measures [34, 57] or convolutional filter grouping [6, 54].”