Simplifying ConvNets for Fast Learning

By Franck Mamalet, Christophe Garcia. 2012

144 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Efficiency & Compression

Citations in the corpus

Xception: Deep Learning with Depthwise Separable Convolutions (2016, background): “While the use of spatially separable convolutions in neural networks has a long history, going back to at least 2012 [12] (but likely even earlier), the depthwise version is more recent.”
Convolutional neural networks at constrained time cost (2014, background): “Deep CNNs not only have been continuously advancing the image classiﬁcation accuracy [14, 21, 24, 1, 9, 22, 23], but also play as generic feature extractors for various recognition tasks such as object detection [6, 9], semantic segmentation [6, 8], and image retrieval [14, 19].”
Speeding up Convolutional Neural Networks with Low Rank Expansions (2014, methodology): “[22] design the network to use rank-1 filters from the outset and combine them with an average pooling layer; however, the technique cannot be applied to general network designs.”
Training CNNs with Low-Rank Filters for Efficient Image Classification (2015, neutral): “layers with filters of differing orientation [3, 2].”
Image Super-Resolution Using Deep Convolutional Networks (2014, background): “or simplifying the trained networks [10], [30], with possible slight degradation in performance.”
Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups (2016, methodology): “filters in the form of low-rank spatial filters [28, 18, 34, 25, 16], using frequencybased convolution [29, 31].”
A Survey of Quantization Methods for Efficient Neural Network Inference (2021, neutral): “…model architectures : One line of work has focused on optimizing the NN model architecture in terms of its micro-architecture [100, 110, 125, 163, 164, 205, 245, 272] (e.g., kernel types such as depth-wise convolution or low-rank factorization) as well as its macro-architecture [99, 100, 103,…”
Interleaved Structured Sparse Convolutional Neural Networks (2018, methodology): “Using a pair of 1×3 and 3× 1 kernels to approximate a 3× 3 kernel [12, 13, 27] is an example of using the product of two small (low-rank) filters.”
MetaFormer is Actually What You Need for Vision (2021, methodology): “8 Pooling → Depthwise Convolution [9, 37] 11.”
MetaFormer Baselines for Vision (2022, methodology): “Although CAFormer is just built by equipping token mixers of separable convolutions [9, 48, 59] in bottom stages and vanilla self-attention [72] in top stages, it already consistently outperforms other models in different sizes, as clearly shown in Figure 3.”
Smoothed dilated convolutions for improved dense prediction (2018, methodology): “In order to achieve this, we propose separable and shared (SS) convolutions, based on separable convolutions [4, 23].”
InceptionNeXt: When Inception Meets ConvNeXt (2023, unknown): “Among these modern CNN models, the common key feature is the large receptive field that is usually achieved by depthwise convolution [41, 7] with large kernel size (e.”
Compressing deep neural networks using a rank-constrained topology (2015, unknown): “, [17, 18] and the references contained therein).”
Survey on semantic segmentation using deep learning techniques (2019, methodology): “Xception [31] Depthwise Separable Convolutions [32] Inception V3 ResNet ImageNet JFT (Google’s) FastEval14k April 4, 2017 YES Residual Network - ResNet [34] is the most popular and widely used neural network for semantic segmentation.”
Axial Residual Networks for CycleGAN-based Voice Conversion (2021, background): “It also resembles spatially separable convolution where the expressivity of one convolution layer is approximated in two steps for computational efficiency [20].”
Interleaved Group Convolutions for Deep Neural Networks (2017, background): “In the spatial extent, small kernels are developed, such as 3 × 3 , 3 × 1 , 1 × 3 [35, 29, 17, 26, 18].”