DCFNet: Deep Neural Network with Decomposed Convolutional Filters
By Qiang Qiu, Xiuyuan Cheng, Robert Calderbank, Guillermo Sapiro. 2018
73 citations recorded by the source corpus.
Filters in a Convolutional Neural Network (CNN) contain model parameters learned from enormous amounts of data. In this paper, we suggest to decompose convolutional filters in CNN as a truncated expansion with pre-fixed bases, namely the Decomposed Convolutional Filters network (DCFNet), where the expansion coefficients remain learned from data. Such a structure not only reduces the number of trainable parameters and computation, but also imposes filter regularity by bases truncation. Through extensive experiments, we consistently observe that DCFNet maintains accuracy for image classification tasks with a significant reduction of model parameters, particularly with Fourier-Bessel (FB) bases, and even with random bases. Theoretically, we analyze the representation stability of DCFNet with respect to input variations, and prove representation stability under generic assumptions on the expansion coefficients. The analysis is consistent with the empirical observations.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Qiang Qiu
- Xiuyuan Cheng
- Guillermo Sapiro
- Robert Calderbank
- Karthik Ramani
- Ze Wang
- Ayan Sinha
- Ramalingam Chellappa
In Computer Vision and Pattern Recognition
Citations in the corpus
- RotDCF: Decomposition of Convolutional Filters for Rotation-Equivariant Deep Networks (2018, methodology): “The factor 2 is usually between and depending on the network and the problem [28].”
- Deep Learning 3D Shapes Using Alt-az Anisotropic 2-Sphere Convolution (2018, background): “(2018); Qiu et al. (2018); Kondor & Trivedi (2018),) and are proven to be effective; but they typically incur a significant increase in the number of parameters and computational load.”
- Continual Learning with Filter Atom Swapping (2022, unknown): “(Qiu et al., 2018) proposes an convolutional filter decomposition as a truncated expansion with pre-fixed filter atoms.”
- Stochastic Conditional Generative Networks with Basis Decomposition (2019, methodology): “Specifically, we apply convolutional filer decomposition as in [20] to write w = ψ a , ψ ∈ R L × L × K , where ψ are basis elements, a are decomposition coefficients, and K is a pre-defined small value, e.g., K = 5 .”