Building Efficient Deep Neural Networks With Unitary Group Convolutions
By Ritchie Zhao, Yuwei Hu, Jordan Dotzel, Christopher De Sa, Zhiru Zhang. 2018
29 citations recorded by the source corpus.
We propose unitary group convolutions (UGConvs), a building block for CNNs which compose a group convolution with unitary transforms in feature space to learn a richer set of representations than group convolution alone. UGConvs generalize two disparate ideas in CNN architecture, channel shuffling (i.e. ShuffleNet) and block-circulant networks (i.e. CirCNN), and provide unifying insights that lead to a deeper understanding of each technique. We experimentally demonstrate that dense unitary transforms can outperform channel shuffling in DNN accuracy. On the other hand, different dense transforms exhibit comparable accuracy performance. Based on these observations we propose HadaNet, a UGConv network using Hadamard transforms. HadaNets achieve similar accuracy to circulant networks with lower computation complexity, and better accuracy than ShuffleNets with the same number of parameters and floating-point multiplies.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Elliot J. Crowley
- Wayne W. C. Luk
- Gavia Gray
- Amos J. Storkey
- Ruizhe Zhao
- Sergey Levine
- Chelsea Finn
- Antreas Antoniou
In Computer Vision and Pattern Recognition
- Elliot J. Crowley
- Wayne W. C. Luk
- Gavia Gray
- Amos J. Storkey
- Ruizhe Zhao
- Dacheng Tao
- Sergey Levine
- Antreas Antoniou
Citations in the corpus
- Efficient Structured Pruning and Architecture Searching for Group Convolution (2018, methodology): “It is not promising regarding the high test error, but this performance is on par with [51], even if they have higher budget in both training Figure 6: Change in ResNet-164 on CIFAR-100 accuracy while reducing number of parameters.”
- Separable Layers Enable Structured Efficient Linear Substitutions (2019, background): “The ACDC layer (Moczulski et al., 2016) was inspired by circulant transforms and subsequent work has focused on the potential of these transforms (Zhao et al., 2019).”