CondenseNet: An Efficient DenseNet Using Learned Group Convolutions

By Gao Huang, Shichen Liu, L. V. D. van der Maaten, Kilian Q. Weinberger. 2017

873 citations recorded by the source corpus.

Deep neural networks are increasingly used on mobile devices, where computational resources are limited. In this paper we develop CondenseNet, a novel network architecture with unprecedented efficiency. It combines dense connectivity with a novel module called learned group convolution. The dense connectivity facilitates feature re-use in the network, whereas learned group convolutions remove connections between layers for which this feature re-use is superfluous. At test time, our model can be implemented using standard group convolutions, allowing for efficient computation in practice. Our experiments show that CondenseNets are far more efficient than state-of-the-art compact convolutional networks such as ShuffleNets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Efficiency & Compression

Citations in the corpus

AutoGrow: Automatic Layer Growing in Deep Convolutional Networks (2019, neutral): “Structure pruning [37, 18, 17, 27, 12, 24, 23, 7, 13, 10] and growing [9, 38, 27, 7] were proposed for different goals, such as efﬁcient inference [37, 18, 17, 12, 24, 23, 7, 13, 10], lifelong learning [38] and model adaptation [9, 27].”
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations (2019, background): “MSDNets bind network layers with DenseNet-like (Huang et al., 2017b) skip connections.”
Dynamic Neural Networks: A Survey (2021, methodology): “Moreover, model compression typically adopts pruning [28] or quantization [29] techniques to produce compact static models, which treat all the inputs in the same way.”
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018, background): “The recent CondenseNet [10] also supports the viewpoint.”
MnasNet: Platform-Aware Neural Architecture Search for Mobile (2018, background): “2018) utilizes low-cost pointwise group convolution and channel shuffle; Condensenet (Huang et al. 2018) learns to connect group convolutions across layers; Recently, MobileNetV2 (Sandler et al.”
A Survey on Neural Architecture Search (2019, background): “To the best of our knowledge, the only work claiming a lower error on this task is the work by Huang et al. (2018b) which claims 1% error.”
Layer-compensated Pruning for Resource-constrained Convolutional Neural Networks (2018, background): “, number of filters in convolution and growth rate in CondenseNet (Huang et al., 2017)).”
Path-Level Network Transformation for Efficient Architecture Search (2018, methodology): “When combined with state-of-the-art human-designed architectures such as DenseNets (Huang et al., 2017b) and PyramidNets (Han et al., 2017), the best discovered cell shows signiﬁcantly improved parameter efﬁciency and better results compared to the original ones.”
Searching for MobileNetV3 (2019, background): “CondenseNet[21] learns group convolutions at the training stage to keep useful dense connections between layers for feature re-use.”
Rethinking the Value of Network Pruning (2018, background): “Note that the connection between network pruning and architecture learning has also been made in prior works (Han et al., 2015; Liu et al., 2017; Gordon et al., 2018; Huang et al., 2018), but to our knowledge we are the first to isolate the effect of inheriting weights and solely compare pruningobta”
Training with Quantization Noise for Extreme Model Compression (2020, methodology): “Many compression methods focus on efficient parameterization, via weight pruning [5, 9, 10, 11], weight sharing [12, 13, 14] or with dedicated architectures [4, 15, 16].”
DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures (2018, methodology): “CondenseNet [18] proposed to use a group convolution [2] variant in order to achieve state-of-the-art computational eﬃciency.”