SPINN: synergistic progressive inference of neural networks over device and cloud
By Stefanos Laskaridis, Stylianos I. Venieris, Mário Almeida, Ilias Leontiadis, Nicholas Donald Lane. ACM/IEEE International Conference on Mobile Computing and Networking, 2020
364 citations recorded by the source corpus.
Despite the soaring use of convolutional neural networks (CNNs) in mobile applications, uniformly sustaining high-performance inference on mobile has been elusive due to the excessive computational demands of modern CNNs and the increasing diversity of deployed devices. A popular alternative comprises offloading CNN processing to powerful cloud-based servers. Nevertheless, by relying on the cloud to produce outputs, emerging mission-critical and high-mobility applications, such as drone obstacle avoidance or interactive applications, can suffer from the dynamic connectivity conditions and the uncertain availability of the cloud. In this paper, we propose SPINN, a distributed inference system that employs synergistic device-cloud computation together with a progressive inference method to deliver fast and robust CNN inference across diverse settings. The proposed system introduces a novel scheduler that co-optimises the early-exit policy and the CNN splitting at run time, in order to adapt to dynamic conditions and meet user-defined service-level requirements. Quantitative evaluation illustrates that SPINN outperforms its state-of-the-art collaborative inference counterparts by up to 2× in achieved throughput under varying network conditions, reduces the server cost by up to 6.8× and improves accuracy by 20.7% under latency constraints, while providing robust operation under uncertain connectivity conditions and significant energy savings compared to cloud-centric execution.
Subfields: Computer Networks and Communications, Computer Vision and Pattern Recognition
View Paper: https://www.semanticscholar.org/paper/f0c962a28e9b988140048c25a912a4fc02f38cdb
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Networks and Communications
In Computer Vision and Pattern Recognition
Citations in the corpus
- Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges (2021, background): “Specifically, these studies [24, 67, 87, 109, 132, 140, 141, 148, 150, 155, 157, 158, 168, 172] define a loss function for each of the classifier, and minimize the weighted sum of cross entropy losses per sample as follows: where 3) by treating the final classifier (last exit) as teacher model and…”