Sequence-to-sequence neural net models for grapheme-to-phoneme conversion

By Kaisheng Yao, Geoffry Zweig. 2015

169 citations recorded by the source corpus.

Sequence-to-sequence translation methods based on generation with a side-conditioned language model have recently shown promising results in several tasks. In machine translation, models conditioned on source side words have been used to produce target-language text, and in image captioning, models conditioned images have been used to generate caption text. Past work with this approach has focused on large vocabulary tasks, and measured quality in terms of BLEU. In this paper, we explore the applicability of such models to the qualitatively different grapheme-to-phoneme task. Here, the input and output side vocabularies are small, plain n-gram models do well, and credit is only given when the output is exactly correct. We find that the simple side-conditioned generation approach is able to rival the state-of-the-art, and we are able to significantly advance the stat-of-the-art with bi-directional long short-term memory (LSTM) neural networks that use the same alignment information that is used in conventional approaches.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Speech & Audio

Citations in the corpus

Deep Voice: Real-time Neural Text-to-Speech (2017, methodology): “After approximately 20,000 iterations, the model converges to a phoneme error rate of 5.8% and a word error rate of 28.7%, which are on par with previous reported results (Yao & Zweig, 2015).”
A Survey on Neural Speech Synthesis (2021, methodology): “Thus, G2P conversion in this kind of languages is mainly responsible for polyphone disambiguation, which decides the appropriate pronunciation based on the current word context [441, 392, 224, 295, 321, 29, 257].”
Neural Source-Filter Waveform Models for Statistical Parametric Speech Synthesis (2019, methodology): “In TTS systems based on a statistical parametric speech synthesis (SPSS) framework [2], [3], deep neural networks have been used to derive linguistic features from text [4].”
The SIGMORPHON 2020 Shared Task on Multilingual Grapheme-to-Phoneme Conversion (2020, background): “…2005, Bisani and Ney 2008, Wu et al. 2014, Novak et al. 2016) or discriminative models based on conditional random ﬁelds (Lehnen et al. 2013), recurrent neural networks (e.g., Rao et al. 2015, Yao and Zweig 2015, van Esch et al. 2016, Lee et al. 2020) or transformers (Yolchuyeva et al. 2019).”
Listen, attend and spell: A neural network for large vocabulary conversational speech recognition (2015, background): “DNNs have also produced significant gains in pronunciation models that map words to phoneme sequences [7, 8].”
Morphological Inflection Generation Using Character Sequence to Sequence Learning (2015, methodology): “Neural encoder-decoder models of string transduction have also been used for sub-word level transformations like grapheme-to-phoneme conversion (Yao and Zweig, 2015; Rao et al., 2015).”
Neural Harmonic-plus-Noise Waveform Model with Trainable Maximum Voice Frequency for Text-to-Speech Synthesis (2019, background): “In text-to-speech (TTS) systems using statistical parametric speech synthesis [1], neural-network (NN)-based models have been introduced to both the front-end text analyzer and the back-end acoustic models [2, 3, 4, 5, 6].”
Knowledge Distillation from Bert in Pre-Training and Fine-Tuning for Polyphone Disambiguation (2019, background): “For alphabetic languages such as English and French, the main challenge of G2P conversion is how to predict the pronunciations of out-of-vocabulary (OOV) words [3, 4, 5].”
Convolutional Sequence to Sequence Model with Non-Sequential Greedy Decoding for Grapheme to Phoneme Conversion (2018, methodology): “Second, we also constructed datasets with stress markings on phonemes, making the task of our study more realistic and difficult than those of previous studies, which removed the stress markings [5, 14, 25].”
A bi-directional LSTM approach for polyphone disambiguation in Mandarin Chinese (2016, methodology): “Our approach is motivated by recent tremendous success of BLSTM models in English G2P conversion [1] [13] and various sequential learning tasks [14, 15, 16, 17].”
Variable length word encodings for neural translation models (2016, methodology): “We attempt to implement LSTM recurrent neural network language models based on Zaremba et al. (2014) for rescoring in the Syntactical-Guided Neural Machine Translation system by Stahlberg et al.”