CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs
By Liangzhen Lai, Naveen Suda, Vikas Chandra. 2018
489 citations recorded by the source corpus.
Deep Neural Networks are becoming increasingly popular in always-on IoT edge devices performing data analytics right at the source, reducing latency as well as energy consumption for data communication. This paper presents CMSIS-NN, efficient kernels developed to maximize the performance and minimize the memory footprint of neural network (NN) applications on Arm Cortex-M processors targeted for intelligent IoT edge devices. Neural network inference based on CMSIS-NN kernels achieves 4.6X improvement in runtime/throughput and 4.9X improvement in energy efficiency.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
Citations in the corpus
- Federated Learning with Non-IID Data (2018, background): “There has been great progress recently on inference on the device [1, 2].”
- A 64-mW DNN-Based Visual Navigation Engine for Autonomous Nano-Drones (2018, methodology): “To compare and validate our experimental results with respect to the current state-of-the-art, we targeted the most efficient CNN implementation currently available for micro-controllers, namely CMSIS-NN [28].”
- Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition (2018, methodology): “CMSISNN[21] covers a new optimized implementation of neural network operations for ARM microcontrollers, and uses Speech Commands to train and evaluate the results.”
- I-BERT: Integer-only BERT Quantization (2021, methodology): “Thus, the complete removal of floating point arithmetic for inference could have a major impact on designing applications, software, and hardware for efficient inference at the edge [6, 30].”
- MCUNet: Tiny Deep Learning on IoT Devices (2020, methodology): “Compared to CMSIS-NN, TinyEngine reduced memory usage by 2.7× and improve inference efficiency by 22% via code generation, as respectively shown in Figures 3 and 4.”
- Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review (2023, methodology): “CMSIS-NN consists of important DL kernels and supports converting pre-trained models and posttraining quantization.”
- Benchmarking TinyML Systems: Challenges and Direction (2020, methodology): “…enables the design, mapping, and deployment of TinyML algorithms including aggressive quantization techniques (Wang et al., 2019), memory aware neural architecture searches (Fedorov et al., 2019), frameworks (Ten-sorFlow), and efficient inference libraries (Lai et al., 2018; Garofalo et al., 2019).”
- A Survey on Federated Learning for Resource-Constrained IoT Devices (2021, unknown): “…in the domain of novel instructions for executing ML in the CPU [188], [189], design of specialized accelerators [190]–[192], optimized software library [193], [194], development of new memory technologies [195], and near data processing to enable large storage using smaller energy budget [196].”
- Visual Wake Words Dataset (2019, background): “Recent papers benchmarking the speed-latency tradeoffs for the keyword-spotting use-case [9] on microcontrollers propose 20–64 KB model with 10-20 million operations per second [22, 23, 24, 25].”
- TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems (2020, methodology): “The optimized versions employ high-performance ARM CMSIS-NN and Cadence libraries (Lai et al., 2018).”
- On-Device Training Under 256KB Memory (2022, methodology): “There are several deep learning systems for tinyML (TF-Micro [5], CMSIS-NN [42], TinyEngine [48], MicroTVM [18], CMix-NN [15], etc.”
- A TinyML Platform for On-Device Continual Learning With Quantized Latent Replays (2021, methodology): “To reshape all convolution operations intomatrix multiplications, the im2col transformation is applied to the activation tensors to reshape them into2D matrix operands [37].”
- Memory-Driven Mixed Low Precision Quantization For Enabling Deep Network Inference On Microcontrollers (2019, methodology): “To boost the performance of this class of MCUs while leveraging the high flexibility of software-programmability, ARM recently released a software library, CMSIS-NN [14], which enabled the efficient computation of deep networks on tiny microcontrollers.”