Visual Wake Words Dataset

By Aakanksha Chowdhery, Pete Warden, Jonathon Shlens, Andrew G. Howard, Rocky Rhodes. 2019

126 citations recorded by the source corpus.

The emergence of Internet of Things (IoT) applications requires intelligence on the edge. Microcontrollers provide a low-cost compute platform to deploy intelligent IoT applications using machine learning at scale, but have extremely limited on-chip memory and compute capability. To deploy computer vision on such devices, we need tiny vision models that fit within a few hundred kilobytes of memory footprint in terms of peak usage and model size on device storage. To facilitate the development of microcontroller friendly models, we present a new dataset, Visual Wake Words, that represents a common microcontroller vision use-case of identifying whether a person is present in the image or not, and provides a realistic benchmark for tiny vision models. Within a limited memory footprint of 250 KB, several state-of-the-art mobile models achieve accuracy of 85-90% on the Visual Wake Words dataset. We anticipate the proposed dataset will advance the research on tiny vision models that can push the pareto-optimal boundary in terms of accuracy versus memory usage for microcontroller applications.

Subfields: Computer Networks and Communications, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Computer Vision and Pattern Recognition

Citations in the corpus

Flower: A Friendly Federated Learning Research Framework (2020, background): “…has been tremendous progress in enabling the execution of deep learning models on mobile and embedded devices to infer user contexts and behaviors (Fromm et al., 2018; Chowdhery et al., 2019; Malekzadeh et al., 2019; Lee et al., 2019; Yao et al., 2019; LiKamWa et al., 2016; Georgiev et al., 2017).”
MCUNet: Tiny Deep Learning on IoT Devices (2020, methodology): “We used 3 datasets as benchmark: ImageNet [14], Visual Wake Words (VWW) [12], and Speech Commands (V2) [44].”
TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems (2020, methodology): “, 2019), visual object detection (Chowdhery et al., 2019), and human-activity recognition (Chavarriaga et al.”
On-Device Training Under 256KB Memory (2022, methodology): “, a microcontroller with only 256KB SRAM), using less than 1/100 of the memory of existing frameworks while matching the accuracy of cloud training+edge deployment for the tinyML application VWW [21].”
Benchmarking TinyML Systems: Challenges and Direction (2020, background): “Some use cases have been proven viable, but have yet to reach end users because they are too new, like visual wake words (Chowdhery et al., 2019).”
MLPerf Tiny Benchmark (2021, background): “The Visual Wakewords challenge [6] tasked submitters with detecting whether at least one person is in an image.”