Machine Learning at Facebook: Understanding Inference at the Edge

By Carole-Jean Wu, David M. Brooks, Kevin Chen, Douglas Chen, Sy Choudhury, Marat Dukhan, Kim M. Hazelwood, Eldad Isaac, Yangqing Jia, Bill Jia and 16 more. 2019

490 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Computer Science Applications, Computer Vision and Pattern Recognition, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Computer Science Applications

In Computer Vision and Pattern Recognition

In Efficiency & Compression

Citations in the corpus

Sustainable AI: Environmental Implications, Challenges and Opportunities (2021, background): “…high wireless communication overheads, sub-optimal training data distribution in individual client devices [114], large degree of system heterogeneity among client edge devices, and highly-fragmented edge device architectures that make system-level optimization signiﬁcantly more challenging [116].”
Cheetah: Optimizing and Accelerating Homomorphic Encryption for Private Inference (2020, methodology): “Ignoring practical fabrication restrictions, a 1173mm 2 chip would process inferences in 101ms, matching CPU performance and meeting real-time requirements, which we assume to be 100ms as used in certain applications [66].”
SPINN: synergistic progressive inference of neural networks over device and cloud (2020, background): “Furthermore, emerging user-facing [80] and missioncritical [37, 42, 66] CNN applications require low-latency processing to ensure high quality of experience (QoE) [4] and safety [11].”
Edge Learning for B5G Networks With Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing (2022, unknown): “training, it is still a huge challenge for edge devices with insufficient computing capability to perform DI frequently [107].”
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing (2019, background): “On the other hand, unlike high-performance discrete GPUs on the cloud data center, there is no dedicated high-bandwidth memory for mobile GPUs on mobile devices [69].”
Serving DNNs like Clockwork: Performance Predictability from the Bottom Up (2020, background): “With the proliferation of machine learning (ML), model inferences are now not only commonplace but increasingly on the critical path of web requests [29,71].”
Accelerometer: Understanding Acceleration Opportunities for Data Center Overheads at Hyperscale (2020, background): “Hence, even if modern inference accelerators [23, 62, 125, 127] were to ofer an infnite inference speedup, the net microservice performance would only improve by 1.”
Federated Learning with Buffered Asynchronous Aggregation (2021, background): “Our observation that execution time on mobile clients exhibits a folded normal distribution is consistent with previous findings [3].”
Papaya: Practical, Private, and Scalable Federated Learning (2021, background): “Compute capabilities of mobile devices in the ﬁeld differ by an order of magnitude (Wu et al., 2019).”
The Architectural Implications of Facebook's DNN-Based Personalized Recommendation (2019, background): “The detailed performance analysis in this paper will enable future work to consider a broader set of solutions to optimize end-to-end personalized recommendation systems currently running in data centers and motivate additional optimization techniques that address challenges specifically for mobile ”
AutoScale: Energy Efficiency Optimization for Stochastic Edge Inference Using Reinforcement Learning (2020, methodology): “As mobile systems become powerful [31], [41], [50], [107], there has been increasing push to execute DNN inference at the edge [10], [25], [42], [53], [55], [65], [106], [107], [111], [121].”
AutoScale: Optimizing Energy Efficiency of End-to-End Edge Inference under Stochastic Variance (2020, methodology): “to execute DNN inference at the edge [8, 22, 36, 44, 46, 55, 89, 90, 92, 98].”