You Look Twice: GaterNet for Dynamic Filter Selection in CNNs

By Zhourong Chen, Yang Li, Samy Bengio, Si Si. 2018

111 citations recorded by the source corpus.

The concept of conditional computation for deep nets has been proposed previously to improve model performance by selectively using only parts of the model conditioned on the sample it is processing. In this paper, we investigate input-dependent dynamic filter selection in deep convolutional neural networks (CNNs). The problem is interesting because the idea of forcing different parts of the model to learn from different types of samples may help us acquire better filters in CNNs, improve the model generalization performance and potentially increase the interpretability of model behavior. We propose a novel yet simple framework called GaterNet, which involves a backbone and a gater network. The backbone network is a regular CNN that performs the major computation needed for making a prediction, while a global gater network is introduced to generate binary gates for selectively activating filters in the backbone network based on each input. Extensive experiments on CIFAR and ImageNet datasets show that our models consistently outperform the original models with a large margin. On CIFAR-10, our model also improves upon state-of-the-art results.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

Dynamic Neural Networks: A Survey (2021, methodology): “For example, BlockDrop [71] and GaterNet [89] use a policy network to adaptively decide the depth and width of a backbone network.”
Conditional Channel Gated Networks for Task-Aware Continual Learning (2020, methodology): “Specifically, our framework relies on separate task-specific classification heads (multihead architecture), and it employs channel-gating [6, 3] in every layer of the (shared) feature extractor.”
Dynamic Slimmable Network (2021, unknown): “In previous dynamic networks [63, 64, 46, 68, 4, 30, 44, 29, 14, 9], the dynamic routing agent and the main network are jointly trained, analogous to jointly optimized NAS methods [48, 7, 66].”
Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach (2021, unknown): “, [43]–[45]), γ(σ(2)) can adaptively prune the redundant dimensions in the encoded feature vector for different σ(2) due to the intrinsic sparsity discussed in Section III.”
Dynamic Channel and Layer Gating in Convolutional Neural Networks (2020, methodology): “GaterNet [5] is a gating architecture that learns a complete convolutional network jointly and in parallel with the original network, and tries to gate individual filters in the base model.”
Batch-shaping for learning conditional channel gated networks (2019, background): “Gaternet (Chen et al., 2018) trains a completely separate network to gate each individual channel of the main network.”
Fractional Skipping: Towards Finer-Grained Dynamic CNN Inference (2020, methodology): “GaterNet (Chen et al. 2018) uses a separate network to calculate the channel activation strategy.”
AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognition (2021, background): “While BlockDrop [58] dynamically selects which layers to execute per sample during inference, GaterNet [9] proposes a gating”
Attention mechanisms in computer vision: A survey (2021, background): “[5], [37], [53], [54], [55], [56], [57], [58] [25], [59], [60] Spatial attention Generate attention mask across spatial domains and use it to select important spatial regions (e.”
Dynamic Network Quantization for Efficient Video Inference (2021, unknown): “Representative meth-ods for image classiﬁcation, dynamically adjust network depth [13, 33, 59, 21, 63], width [66, 7, 20], perform routing [26, 33] or switch resolutions [62].”
AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition (2021, background): “While GaterNet (Chen et al., 2019b) proposes a separate gating network to learn channel-wise binary gates for the backbone network, Channel gating network (Hua et al., 2019) identiﬁes regions in the features that contribute less to the classiﬁcation result, and skips the computation on a subset of…”
Continual Object Detection via Prototypical Task Correlation Guided Gating Mechanism (2022, unknown): “It has been generally used to choose over different channels in dynamic pruning methods [4,8,10,13,19,21,29] and dynamic slimmable network [25].”