MLPerf Tiny Benchmark

By Colby R. Banbury, Vijay Janappa Reddi, P. Torelli, J. Holleman, Nat Jeffries, C. Király, Pietro Montino, David Kanter, S. Ahmed, Danilo Pau and 12 more. NeurIPS Datasets and Benchmarks, 2021

350 citations recorded by the source corpus.

Advancements in ultra-low-power tiny machine learning (TinyML) systems promise to unlock an entirely new class of smart applications. However, continued progress is limited by the lack of a widely accepted and easily reproducible benchmark for these systems. To meet this need, we present MLPerf Tiny, the first industry-standard benchmark suite for ultra-low-power tiny machine learning systems. The benchmark suite is the collaborative effort of more than 50 organizations from industry and academia and reflects the needs of the community. MLPerf Tiny measures the accuracy, latency, and energy of machine learning inference to properly evaluate the tradeoffs between systems. Additionally, MLPerf Tiny implements a modular design that enables benchmark submitters to show the benefits of their product, regardless of where it falls on the ML deployment stack, in a fair and reproducible manner. The suite features four benchmarks: keyword spotting, visual wake words, image classification, and anomaly detection.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

View Paper: https://www.semanticscholar.org/paper/5319040656bc9671fc46012f6f7d1937a1e93e1a

Citations in the corpus

• A compute-in-memory chip based on resistive random-access memory (2022, neutral): “None of the studies have experimentally measured multiple edge AI applications with complexity matching those in MLPerf Tiny, a commonly used benchmark suite for edge AI hardware 28 .”