DeepMon: Mobile GPU-based Deep Learning Framework for Continuous Vision Applications

By Huynh Nguyen Loc, Youngki Lee, Rajesh Krishna Balan. ACM SIGMOBILE International Conference on Mobile Systems, Applications, and Services, 2017

361 citations recorded by the source corpus.

The rapid emergence of head-mounted devices such as the Microsoft Holo-lens enables a wide variety of continuous vision applications. Such applications often adopt deep-learning algorithms such as CNN and RNN to extract rich contextual information from the first-person-view video streams. Despite the high accuracy, use of deep learning algorithms in mobile devices raises critical challenges, i.e., high processing latency and power consumption. In this paper, we propose DeepMon, a mobile deep learning inference system to run a variety of deep learning inferences purely on a mobile device in a fast and energy-efficient manner. For this, we designed a suite of optimization techniques to efficiently offload convolutional layers to mobile GPUs and accelerate the processing; note that the convolutional layers are the common performance bottleneck of many deep learning models. Our experimental results show that DeepMon can classify an image over the VGG-VeryDeep-16 deep learning model in 644ms on Samsung Galaxy S7, taking an important step towards continuous vision without imposing any privacy concerns nor networking cost.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/d289ce63055c10937e5715e940a4bb9d0af7a8c5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

AutoScale: Energy Efficiency Optimization for Stochastic Edge Inference Using Reinforcement Learning (2020, background): “As mobile systems become powerful [31], [41], [50], [107], there has been increasing push to execute DNN inference at the edge [10], [25], [42], [53], [55], [65], [106], [107], [111], [121].”
Deep Learning With Edge Computing: A Review (2019, methodology): “DeepMon [78] combines quantization with caching of results from intermediate layers on GPUs.”
DeepDecision: A Mobile Deep Learning Framework for Edge Video Analytics (2018, methodology): “Hence, any system that performs offloading of real-time scenes must take this delay into account when measuring the accuracy; however, previous works generally compute the accuracy relative to the original time of frame capture [8], [7].”
AutoScale: Optimizing Energy Efficiency of End-to-End Edge Inference under Stochastic Variance (2020, background): “More recently, with the advancements of powerful mobile System-on-Chips (SoCs) [35, 41, 90], there have been increasing pushes to execute DNN inference on the edge mobile devices [8, 22, 36, 44, 46, 55, 89, 90, 92, 98].”
How Is Energy Consumed in Smartphone Deep Learning Apps? Executing Locally vs. Remotely (2019, background): “Although existing papers have extensively investigated how to run CNN models on mobile devices, including model compression of CNNs [11], GPU acceleration [1], and only processing important frames [12], none of these works considered the energy consumption of executing CNNs on smartphones.”
RILOD: near real-time incremental learning for object detection at the edge (2019, background): “educing the computation cost on the edge devices by compressing pre-trained deep learning models [3, 9, 10, 20], designing lightweight neural network architectures [13, 16], or hardware accelerations [15, 37] — these papers have not addressed the incremental learning problem in edge vision. DeepIncr”
Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review (2023, methodology): “DeepMon [162] uses a combination of quantization and caching to speed up the execution by reducing redundant computations of video frames.”
NestDNN: Resource-Aware Multi-Tenant On-Device Deep Learning for Continuous Mobile Vision (2018, methodology): “Although this technique has gained considerable popularity and has been applied to developing state-of-the-art mobile deep learning systems [8, 14, 17, 20, 38], it has a key drawback : since application developers develop their applications independently, the resource-accuracy trade-off of the…”
Efficient Incremental Learning for Mobile Object Detection (2019, background): “…on reducing the computation cost on the mobile devices by compressing pre-trained deep learning models [10, 9, 20, 3], designing lightweight neural network architectures [16, 13], or hardware accelerations [15, 37] — these papers have not addressed the incremental learning problem in mobile vision.”