Glimpse: Continuous, Real-Time Object Recognition on Mobile Devices

By Tiffany Yu-Han Chen, Lenin Ravindranath, Shuo Deng, Paramvir Victor Bahl, Hari Balakrishnan. ACM International Conference on Embedded Networked Sensor Systems, 2015

446 citations recorded by the source corpus.

Glimpse is a continuous, real-time object recognition system for camera-equipped mobile devices. Glimpse captures full-motion video, locates objects of interest, recognizes and labels them, and tracks them from frame to frame for the user. Because the algorithms for object recognition entail significant computation, Glimpse runs them on server machines. When the latency between the server and mobile device is higher than a frame-time, this approach lowers object recognition accuracy. To regain accuracy, Glimpse uses an active cache of video frames on the mobile device. A subset of the frames in the active cache are used to track objects on the mobile, using (stale) hints about objects that arrive from the server from time to time. To reduce network bandwidth usage, Glimpse computes trigger frames to send to the server for recognizing and labeling. Experiments with Android smartphones and Google Glass over Verizon, AT&T, and a campus Wi-Fi network show that with hardware face detection support (available on many mobile devices), Glimpse achieves precision between 96.4% to 99.8% for continuous face recognition, which improves over a scheme performing hardware face detection and server-side recognition without Glimpse's techniques by between 1.8-2.5×. The improvement in precision for face recognition without hardware detection is between 1.6-5.5×. For road sign recognition, which does not have a hardware detector, Glimpse achieves precision between 75% and 80%; without Glimpse, continuous detection is non-functional (0.2%-1.9% precision).

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering

View Paper: https://www.semanticscholar.org/paper/4c6fe7ceb3f23c44881ba0ffa6efc5741c581a3c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Khaled Ben Letaief
2. Xu Chen
3. Jiasi Chen
4. Xukan Ran
5. Liekang Zeng
6. Zhi Zhou
7. En Li
8. Yuanming Shi

In Electrical and Electronic Engineering

1. Khaled Ben Letaief
2. Yuanming Shi
3. Liekang Zeng
4. Jiasi Chen
5. Zhu Han
6. Zhi Zhou
7. Tao Jiang
8. Kai Yang

Citations in the corpus

• Edge Assisted Real-time Object Detection for Mobile Augmented Reality (2019, background): “Glimpse [8] is a continuous object recognition system on the mobile device designed for cloud offload under challenging network conditions.”
• Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing (2019, methodology): “inference with the model of full precision. &DFKHG 5HVXOWV 0RGHO ZLWKIXOO SUHFLVLRQ 0RELOH'HYLFHV (GJH6HUYHU &ORXG'DWDFHQWHU Fig. 15: The process of semantic cache technique Glimpse [92] is a pioneering effort to introduce cache technique to DNN inference task. For an object detect”
• How Is Energy Consumed in Smartphone Deep Learning Apps? Executing Locally vs. Remotely (2019, background): “Although existing papers have extensively investigated how to run CNN models on mobile devices, including model compression of CNNs [11], GPU acceleration [1], and only processing important frames [12], none of these works considered the energy consumption of executing CNNs on smartphones.”
• Computation Offloading Toward Edge Computing (2019, unknown): “Smartphones can directly migrate the execution of object recognition to the cloud, as the paradigm of MCC, or they employ cloudlets to perform the same computation offloading for latency reducing, which is the form of edge computing [69].”
• Communication-Efficient Edge AI: Algorithms and Systems (2020, background): “ [119], [120], [121], [122], [123] Computation Offloading Based Edge Inference Systems Server-based edge inference: Partial data transmission for communication-efficient inference [124], [125], [126], [127] Raw data encoding for communication-efficient inference [128], [129] Cooperative downlink transm”
• DeepDecision: A Mobile Deep Learning Framework for Edge Video Analytics (2018, methodology): “We measure accuracy using the Intersection over Union (IOU) metric, similar to [5]:”
• Reducto: On-Camera Filtering for Resource-Efficient Real-Time Video Analytics (2020, methodology): “To illustrate the limitations of static thresholds, we evaluated Glimpse [24] on 4 random videos in our dataset.”