Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy

By En Li, Zhi Zhou, Xu Chen. 2018

392 citations recorded by the source corpus.

As the backbone technology of machine learning, deep neural networks (DNNs) have have quickly ascended to the spotlight. Running DNNs on resource-constrained mobile devices is, however, by no means trivial, since it incurs high performance and energy overhead. While offloading DNNs to the cloud for execution suffers unpredictable performance, due to the uncontrolled long wide-area network latency. To address these challenges, in this paper, we propose Edgent, a collaborative and on-demand DNN co-inference framework with device-edge synergy. Edgent pursues two design knobs: (1) DNN partitioning that adaptively partitions DNN computation between device and edge, in order to leverage hybrid computation resources in proximity for real-time DNN inference. (2) DNN right-sizing that accelerates DNN inference through early-exit at a proper intermediate DNN layer to further reduce the computation latency. The prototype implementation and extensive evaluations based on Raspberry Pi demonstrate Edgent's effectiveness in enabling on-demand low-latency edge intelligence.

Subfields: Computer Networks and Communications, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

1. Khaled Ben Letaief
2. Jun Zhang
3. Merouane Debbah
4. Yuyi Mao
5. Xu Chen
6. Zhi Zhou
7. Liekang Zeng
8. Min Chen

In Computer Vision and Pattern Recognition

1. Khaled Ben Letaief
2. Xu Chen
3. Jun Zhang
4. Liekang Zeng
5. Xiaofei Wang
6. Zhi Zhou
7. Yiwen Han
8. En Li

Citations in the corpus

• In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning (2018, methodology): “Besides, our work [14] aims to meet the challenge of running DL on resource-constrained mobile devices.”
• Follow Me at the Edge: Mobility-Aware Dynamic Service Placement for Mobile Edge Computing (2018, background): “s point, and available for use by nearby devices. In the paradigm of MEC, as user workload is served by a nearby edge node rather than the remote cloud, the end-to-end latency is significantly reduced [6]. Although the computation capacity of a mobile user is dramatically augmented by edge cloud, a n”
• Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach (2021, background): “To provide an immersive user experience, DNN-based mobile applications need to be performed within the edge of wireless networks, which eliminates the excessive latency incurred by routing data to the Cloud, and is referred to as edge inference [14], [13].”
• Edge Learning for B5G Networks With Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing (2022, positive): “, sparsification, and pruning techniques [104], [105], [106], to enable distributed inference (DI) with improved efficiency and performance.”
• SAFA: A Semi-Asynchronous Protocol for Fast Federated Learning With Low Overhead (2019, background): “The fusion of Edge Computing and Artificial Intelligence (i.e., Edge Intelligence [11], [12]) has emerged as a new focus of research ever since we began to realize the potential benefits of sinking the computation to and outside the edge whilst the increasing capacity of end devices makes it natural”
• Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2019, positive): “co-inference framework [162] for adaptive DNN partitioning and EEoI.”
• All One Needs to Know about Fog Computing and Related Edge Computing Paradigms: A Complete Survey (2018, neutral): “Applications [20, 51, 61, 100, 154, 159, 174, 175, 190, 191, 247, 263, 271, 276, 287, 306, 320, 321, 323, 330–379]”
• Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing (2019, background): “Part of the results of this journal version paper was presented at 2018 ACM SIGCOMM MECOMM [1].”
• A Panda? No, It's a Sloth: Slowdown Attacks on Adaptive Multi-Exit Neural Network Inference (2020, neutral): “In consequence, recent work (Kang et al., 2017; Li et al., 2018; Zhou et al., 2019) utilizes multi-exit architectures for bringing computationally expensive models, e.g. language models (Zhou et al., 2020; Hou et al., 2020), in the cloud to IoT (or mobile) devices.”
• Mobile Edge Intelligence and Computing for the Internet of Vehicles (2019, methodology): “A similar framework, called Edgent, was proposed in [130].”
• Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing (2019, methodology): “Indeed, the marriage of edge computing and AI has given rise to a new research area, namely, “edge intelligence (EI)” or “edge AI” [9], [10].”