SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers

By Igor Fedorov, Ryan P. Adams, Matthew Mattina, Paul N. Whatmough. 2019

180 citations recorded by the source corpus.

The vast majority of processors in the world are actually microcontroller units (MCUs), which find widespread use performing simple control tasks in applications ranging from automobiles to medical devices and office equipment. The Internet of Things (IoT) promises to inject machine learning into many of these every-day objects via tiny, cheap MCUs. However, these resource-impoverished hardware platforms severely limit the complexity of machine learning models that can be deployed. For example, although convolutional neural networks (CNNs) achieve state-of-the-art results on many visual recognition tasks, CNN inference on MCUs is challenging due to severe finite memory limitations. To circumvent the memory challenge associated with CNNs, various alternatives have been proposed that do fit within the memory budget of an MCU, albeit at the cost of prediction accuracy. This paper challenges the idea that CNNs are not suitable for deployment on MCUs. We demonstrate that it is possible to automatically design CNNs which generalize well, while also being small enough to fit onto memory-limited MCUs. Our Sparse Architecture Search method combines neural architecture search with pruning in a single, unified approach, which learns superior models on four popular IoT datasets. The CNNs we find are more accurate and up to $4.35\times$ smaller than previous approaches, while meeting the strict MCU working memory constraint.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Song Han
2. Quoc V. Le
3. Chuang Gan
4. Ji Lin
5. Mingxing Tan
6. Hanxiao Liu
7. Xiaojun Chang
8. Ligeng Zhu

In Computer Vision and Pattern Recognition

1. Song Han
2. Quoc V. Le
3. Luc Van Gool
4. Chuang Gan
5. Dacheng Tao
6. Alan L. Yuille
7. Yi Yang
8. Ji Lin

Citations in the corpus

• A Comprehensive Survey of Neural Architecture Search (2020, background): “In addition, there is some work to consider how to strike a balance between performance and efficiency [116, 119].”
• Benchmarking TinyML Systems: Challenges and Direction (2020, methodology): “Machine learning (ML) inference on the edge is an increasingly attractive prospect due to its potential for increasing energy efficiency (Fedorov et al., 2019), privacy, responsiveness (Zhang et al., 2017), and autonomy of edge devices.”
• Visual Wake Words Dataset (2019, methodology): “Further speedups are potentially possible with the use of binary weights and activa-2 tions, such as XnorNet [26], Trained Ternary Quantization [27] or with sparse architectures [28].”
• MCUNet: Tiny Deep Learning on IoT Devices (2020, background): “There is limited literature [15, 30, 40, 28] that studies machine learning on microcontrollers.”
• Can Weight Sharing Outperform Random Architecture Search? An Investigation With TuNAS (2020, methodology): “Recently, Neural Architecture Search has been used intensively to find architectures that have better tradeoff between accuracy and latency [38, 41, 8, 5, 37], FLOPS [39], power consumption [16], and memory usage [10].”
• On-Device Training Under 256KB Memory (2022, background): “Such a small memory budget is hardly enough for the inference of deep learning models [48, 47, 7, 12, 44, 25, 45, 60], let alone the training, which requires extra computation for the backward and extra memory for intermediate activation [19].”