Convolutional Neural Networks

By Soravit Changpinyo, Mark Sandler, Andrey Zhmoginov. 2017

141 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

Citations in the corpus

What is the State of Neural Network Pruning? (2020, methodology): “In many cases, what is actually used is a custom variation of some VGG model, with removed fully-connected layers (Changpinyo et al., 2017; Luo et al., 2017), smaller fully-connected layers (Lee et al., 2019b), or added dropout or batchnorm (Liu et al., 2017; Lee et al., 2019b; Peng et al., 2018;…”
A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference (2019, background): “Other similar work [2, 38, 6] to our work that reduces or removed the redundant connections or induces sparsity are motivated by the desire to speed up computation.”
To prune, or not to prune: exploring the efficacy of pruning for model compression (2017, methodology): “Within the context of reducing model size by removing redundant connections, several recent works (Anwar et al., 2015; Lebedev and Lempitsky, 2015; Li et al., 2016; Changpinyo et al., 2017) propose techniques to prune and induce sparsity in a structured way, motivated primarily by the desire to”
Learning Efficient Convolutional Networks through Network Slimming (2017, methodology): “[2] introduces sparsity by random deactivating input-output channel-wise connections in convolutional layers before training, which also yields smaller networks with moderate accuracy loss.”
Compressing RNNs for IoT devices by 15-38x using Kronecker Products (2019, methodology): “Our results (section 4) show that popular compression techniques like pruning [6, 19, 56] and low-rank matrix factorization (LMF) [7, 16, 28] can lead to significant loss in accuracy (>3%) when IoT applications are compressed by factors of 15× or more.”
On-Demand Deep Model Compression for Mobile Devices: A Usage-Driven Model Selection Framework (2018, methodology): “• C 3 : decompose conv i using the sparse random technique [7] and we set the sparsity coefficient θ = 0 .”
Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions (2018, methodology): “…CNN compression techniques have been proposed, such as weight compression (Bhattacharya & Lane, 2016) (Han et al., 2016) (Lane et al., 2016) and decomposition (Changpinyo et al., 2017) (Howard et al., 2017) (Wang et al., 2015), and compact architectures (Iandola et al., 2016) (Lin et al., 2014).”
MobileNetV2: Inverted Residuals and Linear Bottlenecks (2018, background): “A substantial amount of work has also been dedicated to changing the connectivity structure of the internal convolutional blocks such as in ShuffleNet [20] or introducing sparsity [21] and others [22].”
EnergyNet: Energy-Efficient Dynamic Inference (2018, background): “Most of them aim to reduce the model size or the number of 28 computations [3, 4, 5, 6, 7, 8, 9, 10, 11, 12].”
Interleaved Group Convolutions for Deep Neural Networks (2017, background): “Channel-wise random sparse connection [2] sparsiﬁes the ﬁlters in the channel extent that every output channel is connected to a small subset of input channels.”
Image Inpainting with Learnable Feature Imputation (2020, unknown): “Convolutional Neural Networks (CNNs) for image inpainting have led to significant progress in the last couple of years [1,24,42].”
Inverted Residuals and Linear Bottlenecks: Mobile Networks for Classification, Detection and Segmentation (2018, background): “A substantial amount of work has also been dedicated to changing the connectivity structure of the internal convolutional blocks such as in ShuffleNet [19] or introducing sparsity [20] and others [21].”
DCFNet: Deep Neural Network with Decomposed Convolutional Filters (2018, background): “Sparse connections in CNNs have been recently studied in (Ioannou et al., 2016; Anwar et al., 2017; Changpinyo et al., 2017).”
Patient Knowledge Distillation for BERT Model Compression (2019, background): “Only a few work are designed for rich structural information such as deep language models (Changpinyo et al., 2017).”