JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services

By Amir Erfan Eshratifar, Mohammad Saeed Abrishami, Massoud Pedram. IEEE Transactions on Mobile Computing, 2018

305 citations recorded by the source corpus.

Deep learning models are being deployed in many mobile intelligent applications. End-side services, such as intelligent personal assistants, autonomous cars, and smart home services often employ either simple local models on the mobile or complex remote models on the cloud. However, recent studies have shown that partitioning the DNN computations between the mobile and cloud can increase the latency and energy efficiencies. In this paper, we propose an efficient, adaptive, and practical engine, JointDNN, for collaborative computation between a mobile device and cloud for DNNs in both inference and training phase. JointDNN not only provides an energy and performance efficient method of querying DNNs for the mobile side but also benefits the cloud server by reducing the amount of its workload and communications compared to the cloud-only approach. Given the DNN architecture, we investigate the efficiency of processing some layers on the mobile device and some layers on the cloud server. We provide optimization formulations at layer granularity for forward- and backward-propagations in DNNs, which can adapt to mobile battery limitations and cloud server load constraints and quality of service. JointDNN achieves up to 18 and 32 times reductions on the latency and mobile energy consumption of querying DNNs compared to the status-quo approaches, respectively.

Subfields: Computer Networks and Communications, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/cbab055052c6098b21c210e945ca17df143273da

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Computer Vision and Pattern Recognition

Citations in the corpus

Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges (2021, background): “[25] Image classification Speech recognition N/A (No task-specific metrics) AlexNet [64] OverFeat [125] NiN [80] VGG-16 [128] ResNet-50 [43] D, E, L”
AutoScale: Optimizing Energy Efficiency of End-to-End Edge Inference under Stochastic Variance (2020, methodology): “As an intermediate stage, many techniques tried to partition DNN inference execution between the cloud and local mobile device [21, 22, 34, 44, 59], based on performance/energy prediction models.”
Model inversion attacks against collaborative inference (2019, methodology): “The second mode is collaborative inference [8, 17, 25, 27, 43].”
SPINN: synergistic progressive inference of neural networks over device and cloud (2020, background): “[47] incorporates lossy compression to minimise the offload transmission overhead. Nevertheless, to yield high performance, the proposed system sacrifices substantial accuracy (i.e. >5%). JointDNN [8] modelled CNN offloading as a graph split problem, but targets only Model Splitter Execution Engi”
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges (2021, methodology): “Lossless compression of the features using Portable Network Graphics (PNG) algorithm [160] is considered in [159].”
AutoScale: Energy Efficiency Optimization for Stochastic Edge Inference Using Reinforcement Learning (2020, methodology): “As an intermediate stage, many techniques tried to partition DNN inference execution between the cloud and the local mobile device [24], [25], [40], [53], [69],”