TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems

By R. David, Jared Duke, Advait Jain, Vijay Janappa Reddi, Nat Jeffries, Jian Li, Nick Kreeger, Ian Nappier, Meghna Natraj, Shlomi Regev and 3 more. arXiv.org, 2020

717 citations recorded by the source corpus.

Deep learning inference on embedded devices is a burgeoning field with myriad applications because tiny embedded devices are omnipresent. But we must overcome major challenges before we can benefit from this opportunity. Embedded processors are severely resource constrained. Their nearest mobile counterparts exhibit at least a 100---1,000x difference in compute capability, memory availability, and power consumption. As a result, the machine-learning (ML) models and associated ML inference framework must not only execute efficiently but also operate in a few kilobytes of memory. Also, the embedded devices' ecosystem is heavily fragmented. To maximize efficiency, system vendors often omit many features that commonly appear in mainstream systems, including dynamic memory allocation and virtual memory, that allow for cross-platform interoperability. The hardware comes in many flavors (e.g., instruction-set architecture and FPU support, or lack thereof). We introduce TensorFlow Lite Micro (TF Micro), an open-source ML inference framework for running deep-learning models on embedded systems. TF Micro tackles the efficiency requirements imposed by embedded-system resource constraints and the fragmentation challenges that make cross-platform interoperability nearly impossible. The framework adopts a unique interpreter-based approach that provides flexibility while overcoming these challenges. This paper explains the design decisions behind TF Micro and describes its implementation details. Also, we present an evaluation to demonstrate its low resource requirement and minimal run-time performance overhead.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Hardware and Architecture

View Paper: https://www.semanticscholar.org/paper/9fd1684e6d163c89bd2b2887ab1b21b89ad10137

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Kai Han
  2. Yunhe Wang
  3. Jianyuan Guo
  4. Yehui Tang
  5. Chang Xu
  6. Chaoting Xu
  7. Dacheng Tao

In Computer Vision and Pattern Recognition

  1. Kai Han
  2. Yunhe Wang
  3. Jianyuan Guo
  4. Yehui Tang
  5. Chang Xu
  6. Chaoting Xu
  7. Dacheng Tao
  8. Chunjing Xu

Citations in the corpus