Interleaved Group Convolutions for Deep Neural Networks

By Ting Zhang, Guo-Jun Qi, Bin Xiao, Jingdong Wang. 2017

88 citations recorded by the source corpus.

In this paper, we present a simple and modularized neural network architecture, named interleaved group convolutional neural networks (IGCNets). The main point lies in a novel building block, a pair of two successive interleaved group convolutions: primary group convolution and secondary group convolution. The two group convolutions are complementary: (i) the convolution on each partition in primary group convolution is a spatial convolution, while on each partition in secondary group convolution, the convolution is a point-wise convolution; (ii) the channels in the same secondary partition come from different primary partitions. We discuss one representative advantage: Wider than a regular convolution with the number of parameters and the computation complexity preserved. We also show that regular convolutions, group convolution with summation fusion, and the Xception block are special cases of interleaved group convolutions. Empirical results over standard benchmarks, CIFAR-$10$, CIFAR-$100$, SVHN and ImageNet demonstrate that our networks are more efficient in using parameters and computation complexity with similar or higher accuracy.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jiashi Feng
2. Dacheng Tao
3. Quoc V. Le
4. Shuicheng Yan
5. Jingdong Wang
6. Yi Yang
7. Alan L. Yuille
8. Chunhua Shen

In Computer Vision and Pattern Recognition

1. Dacheng Tao
2. Luc Van Gool
3. Yi Yang
4. Jiashi Feng
5. Ming-Hsuan Yang
6. Xiaogang Wang
7. Chunhua Shen
8. Shuicheng Yan

Citations in the corpus

• Efficient Structured Pruning and Architecture Searching for Group Convolution (2018, background): “[50, 49, 44] permute by interleaving channels from different groups, which is also called channel shuffle.”
• CondenseNet: An Efficient DenseNet Using Learned Group Convolutions (2017, methodology): “A range of recent studies has explored efficient convolutional networks that can be trained end-to-end [16, 19, 22, 46, 47, 48, 49].”
• CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attention (2021, background): “LSDA, ShuffleNet (Zhang et al., 2018b) and IGCNets (Zhang et al., 2017) devote to reducing the computational cost by approximating the original operations (self-attention and convolution) in a lower cost.”
• ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018, background): “Group convolution is at the core of modern network architectures [33, 35, 12, 34, 31, 26].”
• ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices (2017, background): “However, [41] did not specially investigate the effectiveness of channel shuffle itself and its usage in tiny model design.”
• Quantization Mimic: Towards Very Tiny CNN for Object Detection (2018, methodology): “Beyond these works, Zhang et al.[19] propose a general group convolution algorithm and show that Xception is the special case of their method.”
• Interleaved Structured Sparse Convolutional Neural Networks (2018, methodology): “This consistency observation is different from [43] and might stem from that the balance condition is only used to 1 × 1 group convolutions and that there is no coupling with spatial convolutions.”
• RC-DARTS: Resource Constrained Differentiable Architecture Search (2019, methodology): “The IGCV models [38, 35] introduce novel interleaved group convolutions to improve the representation capablilities of models with the same number of parameters and model complexities.”