Training Keyword Spotting Models on Non-IID Data with Federated Learning
By Andrew Straiton Hard, Kurt Partridge, Cam-Tu Nguyen, Niranjan A. Subrahmanya, Aishanee Shah, Pai Zhu, Ignacio Lopez-Moreno, Rajiv Mathews. 2020
74 citations recorded by the source corpus.
We demonstrate that a production-quality keyword-spotting model can be trained on-device using federated learning and achieve comparable false accept and false reject rates to a centrally-trained model. To overcome the algorithmic constraints associated with fitting on-device data (which are inherently non-independent and identically distributed), we conduct thorough empirical studies of optimization algorithms and hyperparameter configurations using large-scale federated simulations. To overcome resource constraints, we replace memory intensive MTR data augmentation with SpecAugment, which reduces the false reject rate by 56%. Finally, to label examples (given the zero visibility into on-device data), we explore teacher-student training.
Subfields: Artificial Intelligence, Computer Science Applications
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Zachary B. Charles
- Jakub Konecný
- Peter Richtárik
- Deepesh Data
- Suhas N. Diggavi
- Martin Jaggi
- Nicolas Loizou
- Virginia Smith
In Computer Science Applications
Citations in the corpus
- On the Outsized Importance of Learning Rates in Local Update Methods (2020, methodology): “The use of group norm over batch norm in federated learning was first advocated by Hsieh et al. (2019). We perform small amounts of data augmentation and preprocessing, as is standard with CIFAR-100.”