FRILL: A Non-Semantic Speech Embedding for Mobile Devices
By J. Peplinski, Joel Shor, Sachin P. Joglekar, Jake Garrison, Shwetak N. Patel. 2020
29 citations recorded by the source corpus.
Learned speech representations can drastically improve performance on tasks with limited labeled data. However, due to their size and complexity, learned representations have limited utility in mobile settings where run-time performance can be a significant bottleneck. In this work, we propose a class of lightweight non-semantic speech embedding models that run efficiently on mobile devices based on the recently proposed TRILL speech embedding. We combine novel architectural modifications with existing speed-up techniques to create embedding models that are fast enough to run in real-time on a mobile device and ex-hibit minimal performance degradation on a benchmark of non-semantic speech tasks. One such model (FRILL) is 32x faster on a Pixel 1 smartphone and 40% the size of TRILL, with an average decrease in accuracy of only 2%. To our knowledge, FRILL is the highest-quality non-semantic embedding designed for use on mobile devices. Furthermore, we demonstrate that these representations are useful for mobile health tasks such as non-speech human sounds detection and face-masked speech detection. Our models 1 and code 2 are publicly available.
Subfields: Artificial Intelligence, Speech & Audio, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Speech & Audio
In Signal Processing
Citations in the corpus
- BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition (2021, methodology): “Average Accuracy Measure: As in [89], we use the average accuracy measure over all NOSS tasks as metric for quantifying the general quality of an embedding.”