SqueezeWave: Extremely Lightweight Vocoders for On-device Speech Synthesis

By Bohan Zhai, Tianren Gao, Flora Xue, Daniel Rothchild, Bichen Wu, Joseph E. Gonzalez, Kurt Keutzer. 2020

27 citations recorded by the source corpus.

Automatic speech synthesis is a challenging task that is becoming increasingly important as edge devices begin to interact with users through speech. Typical text-to-speech pipelines include a vocoder, which translates intermediate audio representations into an audio waveform. Most existing vocoders are difficult to parallelize since each generated sample is conditioned on previous samples. WaveGlow is a flow-based feed-forward alternative to these auto-regressive models (Prenger et al., 2019). However, while WaveGlow can be easily parallelized, the model is too expensive for real-time speech synthesis on the edge. This paper presents SqueezeWave, a family of lightweight vocoders based on WaveGlow that can generate audio of similar quality to WaveGlow with 61x - 214x fewer MACs. Code, trained models, and generated audio are publicly available at this https URL.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Sheng Zhao
6. Hung-yi Lee
7. Yi Ren
8. Zhou Zhao

In Signal Processing

1. Xu Tan
2. Tao Qin
3. Tie-Yan Liu
4. Frank K. Soong
5. Hung-yi Lee
6. Po-Chun Hsu
7. Yang Zhang
8. Sheng Zhao

Citations in the corpus

• A Survey on Neural Speech Synthesis (2021, methodology): “WaveGAN [396] Voc melS FF−→wav ICASSP20 2019.10 DCA-Tacotron [17] AM ph AR−→melS AR−→wav ICASSP20 2019.10 WaveFlow [265] Voc melS AR−→wav ICML20 2019.12 SqueezeWave [426] Voc melS FF−→wav arXiv20 2020.01 AlignTTS [422] AM ch/ph FF−→melS FF−→wav ICASSP20 2020.03 RobuTrans [190] AM ph AR−→melS AR−→wav”
• HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis (2020, methodology): “The synthesis speed was measured on GPU and CPU environments according to the recent research trends regarding the efficiency of neural networks (Kumar et al., 2019, Zhai et al., 2020, Tan et al., 2020).”
• WG-WaveNet: Real-Time High-Fidelity Speech Synthesis without GPU (2020, methodology): “Inspired by [11], the input of g-20 is reshaped to groups of 20 samples.”
• Fcl-Taco2: Towards Fast, Controllable and Lightweight Text-to-Speech Synthesis (2021, background): “Previous work mostly focused on the optimization of vocoder for edge deployment [3-5], while less attention has been paid to the acoustic model [6].”
• Lightweight LPCNet-Based Neural Vocoder with Tensor Decomposition (2020, methodology): “A similar approach was proposed for SqueezeWave [9], which replaces 1D convolution of WaveGlow [7] with depth-wise convolution and simplifies the conditioning network.”