CondConv: Conditionally Parameterized Convolutions for Efficient Inference

By Brandon Yang, Gabriel Bender, Quoc V. Le, Jiquan Ngiam. 2019

870 citations recorded by the source corpus.

Convolutional layers are one of the basic building blocks of modern deep neural networks. One fundamental assumption is that convolutional kernels should be shared for all examples in a dataset. We propose conditionally parameterized convolutions (CondConv), which learn specialized convolutional kernels for each example. Replacing normal convolutions with CondConv enables us to increase the size and capacity of a network, while maintaining efficient inference. We demonstrate that scaling networks with CondConv improves the performance and inference cost trade-off of several existing convolutional neural network architectures on both classification and detection tasks. On ImageNet classification, our CondConv approach applied to EfficientNet-B0 achieves state-ofthe-art performance of 78.3% accuracy with only 413M multiply-adds. Code and checkpoints for the CondConv Tensorflow layer and CondConv-EfficientNet models are available at: https://github.com/tensorflow/tpu/tree/master/ models/official/efficientnet/condconv.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

In Efficiency & Compression

Citations in the corpus

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding (2020, background): “Prior work (Bapna et al., 2020; Yang et al., 2019; Shazeer et al., 2017) have shown its promising applications in machine translation, language models and computer vision.”
Dynamic Neural Networks: A Survey (2021, methodology): “To improve the representation power without noticeably increasing the computation, soft attention can be performed on multiple convolutional kernels, producing an adaptive ensemble of parameters [13], [19].”
MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers (2020, methodology): “Our mask prediction head is inspired by CondInst [85] and SOLOv2 [92], which extend dynamic convolution [43, 99] to instance segmentation.”
Conditional Convolutions for Instance Segmentation (2020, methodology): “This idea has been explored previously in dynamic filter networks [20] and CondConv [41] mainly for the purpose of increasing the capacity of a classification network.”
Conditional DETR for Fast Training Convergence (2021, background): “CondConv [44] and dynamic convolution [4] mix convolutional kernels with the weights learned from the input.”
Dynamic Convolution: Attention Over Convolution Kernels (2019, methodology): “Compared to the concurrent work [35], our method is more efficient with better performance.”
Omni-Dimensional Dynamic Convolution (2022, methodology): “Recent works (Wang et al., 2017; Hu et al., 2018b; Park et al., 2018; Woo et al., 2018; Yang et al., 2019; Chen et al., 2020) show that incorporating attention mechanisms into convolutional blocks can further push the performance boundaries of modern CNNs, and thus it has attracted great research…”
WeightNet: Revisiting the Design Space of Weight Networks (2020, methodology): “ShuﬄeNetV2 [22] (0.5 × ) 1.4M 41M 39.7 + SE [11] 1.4M 41M 37.5 + SK [16] 1.5M 42M 37.5 + CondConv [38] From the results, we can see SE and CondConv boost the base models of all sizes signiﬁcantly.”
Revisiting Dynamic Convolution via Matrix Decomposition (2021, methodology): “Yang et al. (2019) proposed the use of a sigmoid layer to generate attention scores { π k ( x ) } , leading to a signiﬁcantly large space for the convolution kernel W ( x ) that makes the learning of attention scores { π k ( x ) } difﬁcult.”
Focal Sparse Convolutional Networks for 3D Object Detection (2022, background): “, deformable convolutions [10, 64] and dynamic convolutions [7, 49].”
TAM: Temporal Adaptive Module for Video Recognition (2020, methodology): “Unlike some current dynamic convolution [3, 45], TAM is more flexible, and can directly generate the kernel weights to perform video adaptive convolution.”
Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference (2021, background): “, 2019)) and computer vision ((Yang et al., 2019)) tasks.”
Lite-HRNet: A Lightweight High-Resolution Network (2021, background): “The conditional channel weighting scheme shares the same philosophy to the conditional convolutions [47], dynamic filters [21], and squeeze-excitenetwork [19].”
Dynamic Slimmable Network (2021, neutral): “In previous dynamic networks [63, 64, 46, 68, 4, 30, 44, 29, 14, 9], the dynamic routing agent and the main network are jointly trained, analogous to jointly optimized NAS methods [48, 7, 66].”
Scaling Vision with Sparse Mixture of Experts (2021, methodology): “For computer vision, previous work on MoEs [21, 2, 25, 1, 63, 47, 64] focused on architectures whose scale is considerably smaller than that of both language models and our model.”
Residual Mixture of Experts (2022, methodology): “MoE has been widely used in Natural Language Processing [18, 22, 35, 44] and computer vision [1, 16, 19, 41, 53, 58].”