Learning Dynamic Routing for Semantic Segmentation

By Yanwei Li, Lin Song, Yukang Chen, Zeming Li, X. Zhang, Xingang Wang. 2020

189 citations recorded by the source corpus.

Recently, numerous handcrafted and searched networks have been applied for semantic segmentation. However, previous works intend to handle inputs with various scales in pre-defined static architectures, such as FCN, U-Net, and DeepLab series. This paper studies a conceptually new method to alleviate the scale variance in semantic representation, named dynamic routing. The proposed framework generates data-dependent routes, adapting to the scale distribution of each image. To this end, a differentiable gating function, called soft conditional gate, is proposed to select scale transform paths on the fly. In addition, the computational cost can be further reduced in an end-to-end manner by giving budget constraints to the gating function. We further relax the network level routing space to support multi-path propagations and skip-connections in each forward, bringing substantial network capacity. To demonstrate the superiority of the dynamic property, we compare with several static architectures, which can be modeled as special cases in the routing space. Extensive experiments are conducted on Cityscapes and PASCAL VOC 2012 to illustrate the effectiveness of the dynamic framework. Code is available at https://github.com/yanwei-li/DynamicRouting.

Subfields: Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Dynamic Neural Networks: A Survey (2021, background): “The gating functions take in intermediate features and efficiently produce binary-valued gate vectors to decide: 1) which channels need to be activated [15], [81], [82], [83], [84], 2) which layers need to be skipped [45], [46], [85], [86], 3) which paths should be selected in a SuperNet [101], or 4”
Incremental Few-Shot Semantic Segmentation via Embedding Adaptive-Update and Hyper-class Representation (2022, methodology): “CNN-based methods [23, 29, 50] utilize convolution operations to extract semantic information from feature maps for pixel-level label prediction.”
Image Segmentation Using Deep Learning: A Survey (2020, methodology): “Additional models and methods include BoxSup [119], Graph Convolutional Networks (GCN) [120], Wide ResNet [121], Exfuse [122] (enhancing low-level and high-level features fusion), Feedforward-Net [123], saliency-aware models for geodesic video segmentation [124], Dual Image Segmentation (DIS) [125],”
Multi-Task Reinforcement Learning with Soft Modularization (2020, background): “al [17] alleviate the scale variance in semantic representation with dynamic routing.”
Causal Intervention for Weakly-Supervised Semantic Segmentation (2020, background): “Performance boost is expected if you adopt more advanced ones [27].”
Dynamic Modality Interaction Modeling for Image-Text Retrieval (2021, background): “Different from neural architecture search [20] with static inference, dynamic networks [17, 18, 28, 38] generate execution paths on the fly conditioned on input samples.”
Dynamic Slimmable Network (2021, unknown): “In previous dynamic networks [63, 64, 46, 68, 4, 30, 44, 29, 14, 9], the dynamic routing agent and the main network are jointly trained, analogous to jointly optimized NAS methods [48, 7, 66].”
Rethinking Semantic Segmentation: A Prototype View (2022, background): “Since it was proposed, numerous efforts have been devoted to improving FCN, by, for example, enlarging the receptive field [15, 16, 25, 124, 128, 135]; strengthening context cues [4, 43, 47, 48, 56, 57, 70, 75, 77, 81, 90, 126, 128, 129, 132, 138, 141]; leveraging boundary information [6, 14, 27, 66”
TRAR: Routing the Attention Spans in Transformer for Visual Question Answering (2021, methodology): “6 can be replaced by other controller designs like Gating Function [34].”
Fully Convolutional Networks for Panoptic Segmentation (2020, background): “There is also work to design network architectures for semantic segmentation automatically [30, 26], which is beyond the scope of this paper.”