Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones

By Eric Moulines, Francis Charpentier. 1989

Abstract We review in a common framework several algorithms that have been proposed recently, in order to improve the voice quality of a text-to-speech synthesis based on acoustical units concatenation (Charpentier and Moulines, 1988; Moulines and Charpentier, 1988; Hamon et al., 1989). These algorithms rely on a pitch-synchronous overlap-add (PSOLA) approach for modifying the speech prosody and concatenating speech waveforms. The modifications of the speech signal are performed either in the frequency domain (FD-PSOLA), using the Fast Fourier Transform, or directly in the time domain (TD-PSOLA), depending on the length of the window used in the synthesis process. The frequency domain approach is capable of a great flexibility in modifying the spectral characteristics of the speech signal, while the time domain approach provides very efficient solutions for the real time implementation of synthesis systems. We also discuss the different kinds of distortions involved in these different algorithms.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

WaveNet: A Generative Model for Raw Audio (2016, methodology): “There are two main approaches to realize the speech synthesis part; non-parametric, example-based approach known as concatenative speech synthesis (Moulines & Charpentier, 1990; Sagisaka et al., 1992; Hunt & Black, 1996), and parametric, model-based approach known as statistical parametric speech…”
Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis (2023, background): “Within the scope of text-to-speech (TTS), concatenative synthesis (Moulines & Charpentier, 1990; Hunt & Black, 1996) and statistical parametric synthesis (Yoshimura et al., 1999) were the prevailing approaches.”
A3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing (2022, neutral): “Morrison et al. (2021) propose to retrieve the mod-iﬁed speech segments from other utterances of the same speaker and correct the prosody with a context-aware TD-PSOLA corrector (Moulines & Charpentier, 1990).”
A Survey on Neural Speech Synthesis (2021, methodology): “The early computer-based speech synthesis methods include articulatory synthesis [52, 294], formant synthesis [293, 5, 167, 168], and concatenative synthesis [247, 235, 291, 124, 26].”
WORLD: A Vocoder-Based High-Quality Speech Synthesis System for Real-Time Applications (2016, background): “Other speech synthesis systems have been proposed such as phase vocoder [6], PSOLA [7], and sinusoidal model [8], and high-quality speech synthesis remains a popular research topic.”
Adversarial Audio Synthesis (2018, background): “In concatenative systems, audio is generated by sequencing small, prerecorded portions of speech from a phonetically-indexed dictionary (Moulines & Charpentier, 1990; Hunt & Black, 1996).”
A Review of Deep Learning Based Speech Synthesis (2019, methodology): “There are two different schemes for concatenative synthesis: one is based on linear prediction coefﬁcients (LPCs) [21], the other is based on PSOLA.”
Statistical Parametric Speech Synthesis (2007, methodology): “We can see progress from ﬁ xed inventories, found in diphone systems (Moulines and Charpentier, 1990) to more general, but more resource consuming, techniques of unit-selection synthesis where appropriate sub-word units are automatically selected from large databases of natural speech (Hunt and…”
On the Possible Role of Brain Rhythms in Speech Perception: Intelligibility of Time-Compressed Speech with Periodic and Aperiodic Insertions of Silence (2009, methodology): “Following synthesis, the waveforms were time-compressed by a factor of 3 using a pitch-synchronous, overlap and add (PSOLA) procedure [Moulines and Charpentier, 1990] incorporated into PRAAT, a speech analysis and modification program (http://www.fon.hum.uva.nl/praat/).”
Speech Synthesis Based on Hidden Markov Models (2013, methodology): “In the 1980s, the speech waveform generation component used a small database of phoneme units called ‘‘diphones’’ (the second half of one phone plus the first half of the following) and concatenated them according to the given phoneme sequence by applying signal processing, such as linear predictive”
Distal prosodic context affects word segmentation and lexical processing (2008, methodology): “A series of resynthesized speech stimuli were then derived from these utterances using the pitch-synchronous overlap-and-add (PSOLA) algorithm (Moulines & Charpentier, 1990) as implemented in Praat software (Boersma & Weenink, 2002).”
Speaker-Dependent WaveNet Vocoder (2017, methodology): “One of the representative techniques is concatenative synthesis [1, 2].”
The perception of emotions by ear and by eye (2000, methodology): “All pitch and time manipulations were based on direct waveform manipulations (PSOLA, Charpentier & Moulines, 1989) so that the tokens sounded natural.”
An overlap-add technique based on waveform similarity (WSOLA) for high quality time-scale modification of speech (1993, methodology): “Another form of synchronization is obtained by applying a timedomain pitch-synchronized OLA technique (TD-PSOLA [ 3 ]).”
Real-Time Signal Estimation From Modified Short-Time Fourier Transform Magnitude Spectra (2007, methodology): “Such approaches include the G&L method [1], the synchronized overlap and add algorithm (SOLA) [15], and its various modifications such as WSOLA [16], PAOLA [17], PSOLA [18], the phase vocoder algorithm [19], and some methods for building specific Fig.”
An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning (2020, background): “It decomposes a speech signal into overlapping speech segments [74], each of which represents one of the”
A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI (2023, unknown): “, formant synthesis [40, 41, 95] and concatenative synthesis [29, 69, 75]), statistical parametric speech synthesis (SPSS)-based methods [115, 116, 133], and neural network-based stage [122, 135, 136].”