SRM: A Style-Based Recalibration Module for Convolutional Neural Networks

By Hyunjae Lee, Hyo-Eun Kim, Hyeonseob Nam. 2019

294 citations recorded by the source corpus.

Following the advance of style transfer with Convolutional Neural Networks (CNNs), the role of styles in CNNs has drawn growing attention from a broader perspective. In this paper, we aim to fully leverage the potential of styles to improve the performance of CNNs in general vision tasks. We propose a Style-based Recalibration Module (SRM), a simple yet effective architectural unit, which adaptively recalibrates intermediate feature maps by exploiting their styles. SRM first extracts the style information from each channel of the feature maps by style pooling, then estimates per-channel recalibration weight via channel-independent style integration. By incorporating the relative importance of individual styles into feature maps, SRM effectively enhances the representational ability of a CNN. The proposed module is directly fed into existing CNN architectures with negligible overhead. We conduct comprehensive experiments on general image recognition as well as tasks related to styles, which verify the benefit of SRM over recent approaches such as Squeeze-and-Excitation (SE). To explain the inherent difference between SRM and SE, we provide an in-depth comparison of their representational properties.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Dynamic Neural Networks: A Survey (2021, background): “vide more statistics [125], or replacing FC layers with efficient 1D convolutions [126].”
Single-Source Domain Expansion Network for Cross-Scene Hyperspectral Image Classification (2022, methodology): “Following the common practice [30], [37], [38], we utilize the channel-wise mean and standard deviation of embedding features as style representation, and the difference is that spectral information is used as content representation.”
FcaNet: Frequency Channel Attention Networks (2020, methodology): “Some methods like CBAM [39] and SRM [23] further use global max pooling and global standard deviation pooling to enhance the performance of GAP.”
Efficient Attention Network: Accelerate Attention by Searching Where to Plug (2020, background): “…: the plug-in module extracts internal features of a network by computing their statistics, like mean, variance, or higher-order moments [13, 17]; (2) Processing : the module leverages the extracted features to adaptively generate a mask that measures the importance of the feature maps via…”
Rotate to Attend: Convolutional Triplet Attention Module (2020, result): “While SRM [19] and CBAM were able to obtain marginally better results than triplet attention, our approach is still the lightest in terms of parameters.”
Convolution-Weight-Distribution Assumption: Rethinking the Criteria of Channel Pruning (2020, unknown): “, 2009) SRMNet(Lee et al., 2019) Xavier-normal(Glorot & Bengio, 2010) ImageNet(Russakovsky et al.”
Attention mechanisms in computer vision: A survey (2021, background): “[5], [37], [53], [54], [55], [56], [57], [58] [25], [59], [60] Spatial attention Generate attention mask across spatial domains and use it to select important spatial regions (e.”
SA-Net: Shuffle Attention for Deep Convolutional Neural Networks (2021, background): “…mechanisms most commonly used in computer vision: channel attention and spatial attention , both of which strengthen the original features by aggregating the same feature from all the positions with different aggregation strategies, transformations, and strengthening functions [5, 6, 7, 8, 9].”
Omni-Dimensional Dynamic Convolution (2022, methodology): “To enhance the feature aggregation capability, SRM (Lee et al., 2019) replaces the global average by the channel-wise mean and standard deviation.”
Reducing Domain Gap via Style-Agnostic Networks (2019, methodology): “By explicitly incorporating the styles into CNN representation, SRM can alleviate the inherent the style disparity between domains.”
Reducing Domain Gap by Reducing Style Bias (2021, methodology): “BIN [40] improved classiﬁcation performance by reducing unnecessary style information using trainable normalization, and SRM [28] extended this idea to style-based feature recalibration.”