Focus: Querying Large Video Datasets with Low Latency and Low Cost

By Kevin Hsieh, Ganesh Ananthanarayanan, Peter Bodík, Paramvir Victor Bahl, Matthai Philipose, Phillip B. Gibbons, Onur Mutlu. USENIX Symposium on Operating Systems Design and Implementation, 2018

304 citations recorded by the source corpus.

Large volumes of videos are continuously recorded from cameras deployed for traffic control and surveillance with the goal of answering "after the fact" queries: identify video frames with objects of certain classes (cars, bags) from many days of recorded video. While advancements in convolutional neural networks (CNNs) have enabled answering such queries with high accuracy, they are too expensive and slow. We build Focus, a system for low-latency and low-cost querying on large video datasets. Focus uses cheap ingestion techniques to index the videos by the objects occurring in them. At ingest-time, it uses compression and video-specific specialization of CNNs. Focus handles the lower accuracy of the cheap CNNs by judiciously leveraging expensive CNNs at query-time. To reduce query time latency, we cluster similar objects and hence avoid redundant processing. Using experiments on video streams from traffic, surveillance and news channels, we see that Focus uses 58X fewer GPU cycles than running expensive ingest processors and is 37X faster than processing all the video at query time.

Subfields: Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/583ac84460a98803531aba9d72582af0988875ad

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Nexus: a GPU cluster engine for accelerating DNN-based video analysis (2019, background): “Much work has focused on producing faster models often at small losses in accuracy [2, 31, 41] Further, models of varying accuracy can be combined to maintain high accuracy and performance [6, 14, 17, 20, 35].”
EDF: Ensemble, Distill, and Fuse for Easy Video Labeling (2018, background): “Video stream specialization: Existing work [11, 9, 15] seeks to reduce the time to run expensive CNNs on video by learning cheap video-speciﬁc and camera view speciﬁc models.”
Orca: A Distributed Serving System for Transformer-Based Generative Models (2022, background): “Accordingly, these systems focus on aspects such as batching the executions [7, 16, 35, 42, 56], selecting an appropriate model from multiple model variants [16,27,30,57], deploying multiple models (each for different inference services) on the same device [7, 29, 35, 56], and so on.”
Serving DNNs like Clockwork: Performance Predictability from the Bottom Up (2020, background): “Increasingly, modern ML applications are composed of pipelines or cascades of DNNs [34,43,62].”
Chameleon: scalable adaptation of video analytics (2018, methodology): “Instead, we use an expensive golden configuration to provide the “ground truth”, following prior work [22, 24, 32].”
SPINN: synergistic progressive inference of neural networks over device and cloud (2020, background): “ferent accuracy-latency trade-offs. At run time, they choose the most appropriate variant based on the application requirements and determine where it will be executed. Similarly, classifier cascades [21, 33, 38, 39, 71] require multiple models to obtain performance gains. Despite the advantages of ”
Machine Learning at the Network Edge: A Survey (2019, unknown): “introduced a lowcost, low-latency video querying technique called Focus [45].”
Reducto: On-Camera Filtering for Resource-Efficient Real-Time Video Analytics (2020, background): “This approach is loosely inspired by Focus’s ingest-time processing [36] which targets retrospective queries; we omit Focus’ clustering strategy, which is primarily useful for the tagging queries that Focus targets.”
Orbital Edge Computing: Nanosatellite Constellations as a New Class of Computer System (2020, methodology): “We propose to leverage early discard, which has been studied for search [45], video indexing [44], and drone video processing [97].”