NestDNN: Resource-Aware Multi-Tenant On-Device Deep Learning for Continuous Mobile Vision

By Biyi Fang, Xiao Zeng, Mi Zhang. ACM/IEEE International Conference on Mobile Computing and Networking, 2018

305 citations recorded by the source corpus.

Mobile vision systems such as smartphones, drones, and augmented-reality headsets are revolutionizing our lives. These systems usually run multiple applications concurrently and their available resources at runtime are dynamic due to events such as starting new applications, closing existing applications, and application priority changes. In this paper, we present NestDNN, a framework that takes the dynamics of runtime resources into account to enable resource-aware multi-tenant on-device deep learning for mobile vision systems. NestDNN enables each deep learning model to offer flexible resource-accuracy trade-offs. At runtime, it dynamically selects the optimal resource-accuracy trade-off for each deep learning model to fit the model's resource demand to the system's available runtime resources. In doing so, NestDNN efficiently utilizes the limited resources in mobile vision systems to jointly maximize the performance of all the concurrently running applications. Our experiments show that compared to the resource-agnostic status quo approach, NestDNN achieves as much as 4.2% increase in inference accuracy, 2.0× increase in video frame processing rate and 1.7× reduction on energy consumption.

Subfields: Computer Networks and Communications, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/cb6a82115777edee8d72feeb77427265b2a48e27

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

1. Xu Chen
2. Zhi Zhou
3. Liekang Zeng
4. Xiaofei Wang
5. Yiwen Han
6. Dusist Niyato
7. En Li
8. Junshan Zhang

In Computer Vision and Pattern Recognition

1. Xu Chen
2. Liekang Zeng
3. Xiaofei Wang
4. Zhi Zhou
5. Yiwen Han
6. En Li
7. Dusist Niyato
8. Victor C. M. Leung

Citations in the corpus

• FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout (2021, methodology): “Finally, other model compression methods (Fang et al., 2018; Wang et al., 2019a; Dudziak et al., 2019) remain orthogonal to FjORD.”
• Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing (2019, methodology): “Taking the dynamics of runtime resources into consideration, NestDNN [107] is proposed to offering flexible resource–accuracy tradeoffs for each DNN model.”
• Deep Learning in Mobile and Wireless Networking: A Survey (2018, methodology): “[531] design a framework called NestDNN, to provide flexible resource-accuracy trade-offs on”
• Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2019, methodology): “With the model pruning and recovery scheme, NestDNN [105] transforms the DL model into a set of descendant models, in which the descendant model with fewer resource requirements shares its model parameters with the descendant model requiring more resources, making itself nested inside the descendent”