Normalization Driven Zero-Shot Multi-Speaker Speech Synthesis

By Neeraj Kumar, Srishti Goel, Ankur Narang, Brejesh Lall. 2021

15 citations recorded by the source corpus.

In this paper, we present a novel zero-shot multi-speaker speech synthesis approach (ZSM-SS) that leverages the normalization architecture and speaker encoder with non-autoregressive multi-head attention driven encoder-decoder architecture.Given an input text and a reference speech sample of an unseen person, ZSM-SS can generate speech in that person’s style in a zero-shot manner. Additionally, we demonstrate how the affine parameters of normalization help in capturing the prosodic features such as energy and fundamental frequency in a disentangled fashion and can be used to generate morphed speech output. We demonstrate the efficacy of our proposed architecture on multi-speaker VCTK[1] and LibriTTS [2] datasets, using multiple quantitative metrics that measure generated speech distortion and MOS, along with speaker embedding analysis of the proposed speaker encoder model.

Subfields: Artificial Intelligence, Signal Processing

Citations in the corpus

• YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone (2021, neutral): “ZSM-SS [11] is a Transformer-based architecture with a normalization architecture and an external speaker encoder based on Wav2vec 2.0 [12].”