In Other News: a Bi-style Text-to-speech Model for Synthesizing Newscaster Voice with Limited Data

By Nishant Prateek, Mateusz Lajszczak, Roberto Barra-Chicote, Thomas Drugman, Jaime Lorenzo-Trueba, Thomas Merritt, Srikanth Ronanki, Trevor Wood. 2019

30 citations recorded by the source corpus.

Neural text-to-speech synthesis (NTTS) models have shown significant progress in generating high-quality speech, however they require a large quantity of training data. This makes creating models for multiple styles expensive and time-consuming. In this paper different styles of speech are analysed based on prosodic variations, from this a model is proposed to synthesise speech in the style of a newscaster, with just a few hours of supplementary data. We pose the problem of synthesising in a target style using limited data as that of creating a bi-style model that can synthesise both neutral-style and newscaster-style speech via a one-hot vector which factorises the two styles. We also propose conditioning the model on contextual word embeddings, and extensively evaluate it against neutral NTTS, and neutral concatenative-based synthesis. This model closes the gap in perceived style-appropriateness between natural recordings for newscaster-style of speech, and neutral speech synthesis by approximately two-thirds.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data (2024, background): “The movie that De Moya who was recently awarded the lifetime achievement award starred in 2022 was a box-office hit, despite the mixed reviews.”
Using Vaes and Normalizing Flows for One-Shot Text-To-Speech Synthesis of Expressive Speech (2019, unknown): “For our proposed architecture, we extend a state-of-art sequence-to-sequence NTTS [2, 12, 13] system.”
BOFFIN TTS: Few-Shot Speaker Adaptation by Bayesian Optimization (2020, methodology): “Our base model (the model we adapt to target speakers) is a Tacotron2 [17] style multi-speaker system explained in detail by [1, 18], consisting of an acoustic context-generation model and neural vocoder.”