Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net

By Xingang Pan, Ping Luo, Jianping Shi, Xiaoou Tang. 2018

943 citations recorded by the source corpus.

Convolutional neural networks (CNNs) have achieved great successes in many computer vision problems. Unlike existing works that designed CNN architectures to improve performance on a single task of a single domain and not generalizable, we present IBN-Net, a novel convolutional architecture, which remarkably enhances a CNN’s modeling ability on one domain (e.g. Cityscapes) as well as its generalization capacity on another domain (e.g. GTA5) without finetuning. IBN-Net carefully integrates Instance Normalization (IN) and Batch Normalization (BN) as building blocks, and can be wrapped into many advanced deep networks to improve their performances. This work has three key contributions. (1) By delving into IN and BN, we disclose that IN learns features that are invariant to appearance changes, such as colors, styles, and virtuality/reality, while BN is essential for preserving content related information. (2) IBN-Net can be applied to many advanced deep architectures, such as DenseNet, ResNet, ResNeXt, and SENet, and consistently improve their performance without increasing computational cost. (3) When applying the trained networks to new domains, e.g. from GTA5 to Cityscapes, IBN-Net achieves comparable improvements as domain adaptation methods, even without using data from the target domain. With IBN-Net, we won the 1st place on the WAD 2018 Challenge Drivable Area track, with an mIoU of 86.18%.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

In Model Architectures

Citations in the corpus

Style Normalization and Restitution for Generalizable Person Re-Identification (2020, methodology): “However, IN inevitably removes some discriminative information and results in weaker discrimination capability [32].”
Cluster Contrast for Unsupervised Person Re-Identification (2021, methodology): “Compared with IBN-ResNet-50 and ResNet-50 backbones.”
FastReID: A Pytorch Toolbox for General Instance Re-identification (2020, neutral): “The pooling layer takes X ∈ R 𝑊 × 𝐻 × 𝐶 as input and produces a vector f ∈ R 1 × 1 × 𝐶 as an output of the pooling process, where 𝑊,𝐻,𝐶 respectively represent the width, the height and the channel of the feature maps.”
HINet: Half Instance Normalization Network for Image Restoration (2021, methodology): “Normalization is widely used in high-level computer vision tasks: Batch Normalization [18] and IBN [33] in classiﬁcation [28], Layer Normalization [4] in DETR [6] and GroupNorm [47] in FCOS [40] for detection etc .”
SSN: Learning Sparse Switchable Normalization via SparsestMax (2019, background): “All of them show that normalization methods make great contribution to stabilizing the training and boosting the performance of DNN. Recent study of IBN [19] shows that the hybrid of multiple normalizers in the neural networks can greatly strengthen the generalization ability of DNN.”
Generalizing to Unseen Domains: A Survey on Domain Generalization (2021, background): “[185] introduced Instance Normalization (IN) layers to CNNs to improve the generalization capability”
RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening (2021, methodology): “…on the synthetic data such as GTAV [47] and SYNTHIA [49], numerous DA studies [43, 60, 53, 6, 69, 18, 58, 37, 65] have been proposed in semantic segmentation, but only a few DG studies [64, 44] address semantic segmentation, as the majority of the DG methods mainly focused on image classiﬁcation.”
FSDR: Frequency Space Domain Randomization for Domain Generalization (2021, methodology): “The evaluations were performed over the official validation data of the three targetdomain datasets as in [71, 42, 80, 61, 67, 41].”
Towards Out-Of-Distribution Generalization: A Survey (2021, neutral): “…[38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49] [37], [39], [50], [51], [51], [52], [53], [53], [54], [54], [55], [56], [57], [58], [59], [60] [6], [61], [62], [63], [63], [64], [65], [66], [67], [68], [69], [70](Section 4.1.1) [52], [71], [72], [73], [74], [75], [76],…”
Unsupervised Domain Adaptation Through Synthesis For Person Re-Identification (2020, background): “[13] in 2018, which focuses on enhancing both learning and generalization capacities of deep networks in domain adaptation tasks, such as image recognition and seC on v 1x 1”
A Strong Baseline and Batch Normalization Neck for Deep Person Re-Identification (2019, background): “IBN-Net50 [48], which replaces the BN layers with instance BN layers for ResNet50, is also effective for our baseline.”
DLOW: Domain Flow for Adaptation and Generalization (2018, background): “Recently, there is an increasing interest to apply domain adaptation techniques for semantic segmentation from synthetic data to the real scenario [22, 21, 7, 67, 40, 25, 11, 46, 51, 53, 23, 47, 62, 54, 43, 50, 52, 66, 5].”
Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization Without Accessing Target Domain Data (2019, methodology): “IBN-Net improves domain generalization by fine-tuning the ResNet building blocks.”
Open Compound Domain Adaptation (2019, methodology): “Following [47, 58, 35], a pre-trained VGG-16 [44] is the backbone network for CDriving.”
Domain Generalization: A Survey (2021, methodology): “[57] suggested that the divergence between images of different domains mainly occurs in shallow CNN layers where IN should be inserted.”