RNN Approaches to Text Normalization: A Challenge

By Richard Sproat, Navdeep Jaitly. 2016

120 citations recorded by the source corpus.

This paper presents a challenge to the community: given a large corpus of written text aligned to its normalized spoken form, train an RNN to learn the correct normalization function. We present a data set of general text where the normalizations were generated using an existing text normalization component of a text-to-speech system. This data set will be released open-source in the near future. We also present our own experiments with this data set with a variety of different RNN architectures. While some of the architectures do in fact produce very good results when measured in terms of overall accuracy, the errors that are produced are problematic, since they would convey completely the wrong message if such a system were deployed in a speech application. On the other hand, we show that a simple FST-based filter can mitigate those errors, and achieve a level of accuracy not achievable by the RNN alone. Though our conclusions are largely negative on this point, we are actually not arguing that the text normalization problem is intractable using an pure RNN approach, merely that it is not going to be something that can be solved merely by having huge amounts of annotated text data and feeding that to a general RNN model. And when we open-source our data, we will be providing a novel data set for sequence-to-sequence modeling in the hopes that the the community can find better solutions. The data used in this work have been released and are available at: this https URL

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tao Qin
2. Tie-Yan Liu
3. Xu Tan
4. Yu Zhang
5. Yonghui Wu
6. Frank K. Soong
7. Ye Jia
8. Hung-yi Lee

In Natural Language Processing

1. Tao Qin
2. Tie-Yan Liu
3. Felix Stahlberg
4. Xu Tan
5. Yu Zhang
6. Yonghui Wu
7. Shinji Watanabe
8. Graham Neubig

Citations in the corpus

• ShapeWorld - A new test methodology for multimodal language understanding (2017, background): “Deep networks for language tasks may exhibit similarly odd behavior (Sproat and Jaitly, 2016; Arthur et al., 2016).”
• Tacotron: Towards End-to-End Speech Synthesis (2017, methodology): “We perform simple text normalization, though recent advancements in learned text normalization [26] may render this unnecessary in the future.”
• A Survey on Neural Speech Synthesis (2021, methodology): “TTS Text Analysis Char→Linguistic TN [310, 223, 430], G2P [403, 320, 321] Prosody Prediction [312, 137, 277, 251] Unified Model [252, 437] DeepVoice 1/2 [8, 84]”
• SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training (2021, result): “We consider the English task data from Sproat & Jaitly (2016) and compare our results to those of Stahlberg & Kumar (2020), known to be previously state-of-the-art.”
• Neural Text Normalization with Subword Units (2019, methodology): “Researchers hope a vast amount of such data can counteract the errors inherited in WFST-based models.”
• SPGISpeech: 5, 000 hours of transcribed financial audio for fully formatted end-to-end speech recognition (2021, background): “[4, 5, 6, 7, 8, 9]), and improvements to one model’s performance may degrade end-to-end performance as a whole due to distribution shift [10].”
• Text normalization with convolutional neural networks (2018, methodology): “In order to foster more research in this direction, a challenge with an open-source dataset was published (Sproat and Jaitly 2016).”
• Tacotron: A Fully End-to-End Text-To-Speech Synthesis Model (2017, methodology): “We perform simple text normalization, though recent advancements in learned text normalization (Sproat & Jaitly, 2016) may render this unnecessary in the future.”
• Neural Models of Text Normalization for Speech Applications (2019, methodology): “Sproat and Jaitly (2016) explore two neural models of text normalization.”
• Neural Machine Translation: A Review (2019, background): “recognition [84, 85], cross-lingual word-to-phone alignment [86], bioinformatics [87], text summarization [88], text normalization [89], grammatical error correction [90], question answering [91–93], natural language understanding and inference [62, 94–96], uncertainty detection [97], photo optical ”
• Seq2Edits: Sequence Transduction Using Span-level Edit Operations (2020, methodology): “Our model is competitive across all of these tasks, and improves the state-of-the-art on text normalization (Sproat and Jaitly, 2016), sentence splitting & rephrasing (Botha et al.”
• A Hybrid Text Normalization System Using Multi-Head Self-Attention For Mandarin (2019, methodology): “Different seq2seq models with bi-LSTM, biGRU with attention are proposed in [10, 11].”