On-Demand Deep Model Compression for Mobile Devices: A Usage-Driven Model Selection Framework

By Sicong Liu, Yingyan Lin, Zimu Zhou, Kaiming Nan, Hui Liu, Junzhao Du. 2018

225 citations recorded by the source corpus.

Recent research has demonstrated the potential of deploying deep neural networks (DNNs) on resource-constrained mobile platforms by trimming down the network complexity using different compression techniques. The current practice only investigate stand-alone compression schemes even though each compression technique may be well suited only for certain types of DNN layers. Also, these compression techniques are optimized merely for the inference accuracy of DNNs, without explicitly considering other application-driven system performance (e.g. latency and energy cost) and the varying resource availabilities across platforms (e.g. storage and processing capability). In this paper, we explore the desirable tradeoff between performance and resource constraints by user-specified needs, from a holistic system-level viewpoint. Specifically, we develop a usage-driven selection framework, referred to as AdaDeep, to automatically select a combination of compression techniques for a given DNN, that will lead to an optimal balance between user-specified performance goals and resource constraints. With an extensive evaluation on five public datasets and across twelve mobile devices, experimental results show that AdaDeep enables up to 9.8x latency reduction, 4.3x energy efficiency improvement, and 38x storage reduction in DNNs while incurring negligible accuracy loss. AdaDeep also uncovers multiple effective combinations of compression techniques unexplored in existing literature.

Subfields: Computer Networks and Communications, Electrical and Electronic Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Electrical and Electronic Engineering

Citations in the corpus

AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks (2020, positive): “The AGD framework adopts the differential search algorithm (Liu et al., 2018a).”
AutoScale: Energy Efficiency Optimization for Stochastic Edge Inference Using Reinforcement Learning (2020, methodology): “To execute DNN inference entirely at the edge, many optimizations have been proposed, including model architecture search [95], [104], [110], [124], quantization [15], [30], [52], [62], [65], [111], [119], weight compression [18], [43], [68], [72], and graph pruning [112], [118].”
Deep Learning With Edge Computing: A Review (2019, methodology): “Adadeep [77] automatically chooses between different compression techniques, including pruning and the special filter structures borrowed from MobileNet and SqueezeNet, to meet application requirements and satisfy mobile resource constraints.”
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2019, methodology): “AdaDeep [149] explores the desirable balance between performance and resource constraints, and based on DRL, automatically selects various compression techniques (such as model pruning) to form a compressed model according to current available resources, thus fully utilizing the advantages of them.”
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing (2019, methodology): “To this end, the proposed automatic optimization framework AdaDeep [75] systematically formulates the goals and constraints on accuracy, latency, storage, and energy into a unified optimization problem and leverages DRL to effectively find a good combination of compression techniques.”
Deep Learning in Mobile and Wireless Networking: A Survey (2018, background): “[533] develop a usage-driven framework named AdaDeep, to select a combination of compression techniques for a specific deep NN on mobile platforms.”
Drawing early-bird tickets: Towards more efficient training of deep networks (2019, unknown): “Popular means include pruning (Li et al., 2017; Liu et al., 2017; He et al., 2018; Wen et al., 2016; Luo et al., 2017; Liu et al., 2018a), weight factorization (Denton et al., 2014), weight sharing (Wu et al., 2018a), quantization (Hubara et al., 2017), dynamic inference (Wang et al., 2018b; 2019b;…”
Edge Assisted Real-time Object Detection for Mobile Augmented Reality (2019, background): “There have been extensive works on how to efficiently run these CNN models on mobile devices [35–42].”
A comprehensive survey on model compression and acceleration (2020, background): “If the number of parameters are more in the trained model, then during inference it will take more time to process the input, require more energy and space compared to a small network having less number of parameters (Liu et al. 2018; Han et al. 2015).”
AutoScale: Optimizing Energy Efficiency of End-to-End Edge Inference under Stochastic Variance (2020, methodology): “To fully execute DNN inference at the edge, many optimizations, such as model architecture search [80, 87, 101], quantization [13,26,43,53,55,92,97], weight compression [15, 37, 58, 61] and graph pruning [93, 96], have been proposed.”