Knowledge Adaptation for Efficient Semantic Segmentation

By Tong He, Chunhua Shen, Zhi Tian, Dong Gong, Changming Sun, Youliang Yan. 2019

250 citations recorded by the source corpus.

Both accuracy and efficiency are of significant importance to the task of semantic segmentation. Existing deep FCNs suffer from heavy computations due to a series of high-resolution feature maps for preserving the detailed knowledge in dense estimation. Although reducing the feature map resolution (i.e., applying a large overall stride) via subsampling operations (e.g., polling and convolution striding) can instantly increase the efficiency, it dramatically decreases the estimation accuracy. To tackle this dilemma, we propose a knowledge distillation method tailored for semantic segmentation to improve the performance of the compact FCNs with large overall stride. To handle the inconsistency between the features of the student and teacher network, we optimize the feature similarity in a transferred latent domain formulated by utilizing a pre-trained autoencoder. Moreover, an affinity distillation module is proposed to capture the long-range dependency by calculating the non local interactions across the whole image. To validate the effectiveness of our proposed method, extensive experiments have been conducted on three popular benchmarks: Pascal VOC, Cityscapes and Pascal Context. Built upon a highly competitive baseline, our proposed method can improve the performance of a student network by 2.5% (mIOU boosts from 70.2 to 72.7 on the cityscapes test set) and can train a better compact model with only 8% float operations (FLOPS) of a model that achieves comparable performances.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

FCOS: Fully Convolutional One-Stage Object Detection (2019, background): “Recently, fully convolutional networks (FCNs) [20] have achieved tremendous success in dense prediction tasks such as semantic segmentation [20, 28, 9, 19], depth estimation [17, 31], keypoint detection [3] and counting [2].”
Knowledge Distillation: A Survey (2020, background): “…2018; Wang et al. 2018b, 2019c; Duong et al. 2019; Wu et al. 2020; Wang et al. 2017; Zhang et al. 2020b; Wang et al. 2020b), image/video segmentation (He et al. 2019; Mullapudi et al. 2019; Dou et al. 2020; Liu et al. 2019h; Siam et al. 2019; Hou et al. 2020; Bergmann et al. 2020), action…”
Conditional Convolutions for Instance Segmentation (2020, background): “In computer vision, the closest task to instance segmentation is semantic segmentation, for which fully convolutional networks (FCNs) have shown dramatic success [8, 17, 26, 27, 36].”
Intra-class Feature Variation Distillation for Semantic Segmentation (2020, methodology): “Motivated by this, in [18,24], the authors propose to transfer the densely pairwise relations computed in the feature space.”
Decoders Matter for Semantic Segmentation: Data-Dependent Decoding Enables Flexible Feature Aggregation (2019, unknown): “Typically in semantic segmentation [4, 5, 21, 33, 13], the training loss function is formulated as:”
Enforcing Geometric Constraints of Virtual Normal for Depth Prediction (2019, background): “Recently, owing to the structural features from very deep convolution neural network, such as ResNet [18], various pixel-wise vision tasks have been promoted significantly, such as semantic segmentation [42, 43, 19].”
Diverse Knowledge Distillation for End-to-End Person Search (2020, methodology): “It is a common practice to improve the main network using the additional guidance from external networks (Li, Jin, and Yan 2017; Cheng et al. 2018; Zhu et al. 2019; Hinton, Vinyals, and Dean 2015; Tung and Mori 2019; Liu et al. 2020b,a; Yuan et al. 2020; He et al. 2019).”
FasterSeg: Searching for Faster Real-time Semantic Segmentation (2019, background): “More works target on segmentation efﬁciency in terms of computation cost (He et al., 2019; Marin et al., 2019) and memory usage (Chen et al., 2019).”
FCOS: A Simple and Strong Anchor-Free Object Detector (2020, background): “mantic segmentation [10], [11], [12], depth estimation [13], [14], keypoint detection [15] and counting.”
Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels (2020, unknown): “Current stateof-the-art methods try to further increase the information transfer from the teacher DNN by incorporating various intermediate feature representations and novel loss formulations during training [22], [23], [24], [25], [26].”
Channel-wise Knowledge Distillation for Dense Prediction* (2020, methodology): “The pairwise affinity [24, 13, 23] is employed to transfer the similarity between pixel pairs.”
Collaborative Distillation in the Parameter and Spectrum Domains for Video Action Recognition (2020, background): “He et al. (He et al. 2019) handle the inconsistent features through modeling the feature similarity in a transferred latent domain to capture the long-range dependencies.”