Channel-Level Acceleration of Deep Face Representations

By Adam Polyak, Lior Wolf. 2015

177 citations recorded by the source corpus.

A major challenge in biometrics is performing the test at the client side, where hardware resources are often limited. Deep learning approaches pose a unique challenge: while such architectures dominate the field of face recognition with regard to accuracy, they require elaborate, multi-stage computations. Recently, there has been some work on compressing networks for the purpose of reducing run time and network size. However, it is not clear that these compression methods would work in deep face nets, which are, generally speaking, less redundant than the object recognition networks, i.e., they are already relatively lean. We propose two novel methods for compression: one based on eliminating lowly active channels and the other on coupling pruning with repeated use of already computed elements. Pruning of entire channels is an appealing idea, since it leads to direct saving in run time in almost every reasonable architecture.

Subfields: Computer Vision and Pattern Recognition, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Signal Processing

Citations in the corpus

AMC: AutoML for Model Compression and Acceleration on Mobile Devices (2018, background): “Channel pruning [40, 24, 1, 38] removes the redundant channels from feature maps.”
Channel Pruning for Accelerating Very Deep Neural Networks (2017, neutral): “Inference-time channel pruning is challenging, as reported by previous works [2, 40].”
Structured Pruning of Deep Convolutional Neural Networks (2015, background): “A recently published work induces channel wise sparsity in a network [13].”
BlockDrop: Dynamic Inference Paths in Residual Networks (2017, methodology): “The need to deploy top-performing deep neural network models on mobile devices motivates techniques that can effectively reduce the storage and computational costs of such networks, including knowledge distillation [19, 40, 4], low-rank factorization [25, 47, 41], filter pruning [30, 36, 32, 57], qu”
Compact Deep Convolutional Neural Networks With Coarse Pruning (2016, methodology): “A recently published work propose sparsity at a higher granularity and induce channel level sparsity in a CNN network for deep face application Polyak & Wolf (2015).”
Coreset-Based Neural Network Compression (2018, methodology): “These approaches have been modified in the works of Polyak et al.[45] and Luo et al.”
Pruning and Quantization for Deep Neural Network Acceleration: A Survey (2021, methodology): “The reduce and reused (also described as outbound) method [185] prunes entire filters by computing the statistical variance of each filter’s output using a calibration set.”
ADC: Automated Deep Compression and Acceleration with Reinforcement Learning (2018, background): “Channel pruning [47, 25, 2, 45] removed redundant channels on feature maps.”
Pruning Filters for Efficient ConvNets (2016, methodology): “The activation-based feature map pruning method removes the feature maps with weak activation patterns and their corresponding filters and kernels (Polyak & Wolf (2015)), which needs sample data as input to determine which feature maps to prune.”
Dynamic Recursive Neural Network (2019, methodology): “Related techniques focus on many fields like distillation [3, 15], filter pruning [5, 31], low-rank fac torization [21], quantization [12], compression with struc tured matrices [4, 35] and network binarization [32].”
Recent Advances in Convolutional Neural Network Acceleration (2018, methodology): “ layer. Polyak et al. propose a channel-pruning based method Inbound Prune to compress a redundant network. Their experiment is taken on the platform of Samsung Galaxy S6 and it achieves 1:59 speedup [63]. Recently, pruning is combined with other acceleration techniques to achieve speedup. For examp”