s-Transformer: Segment-Transformer for Robust Neural Speech Synthesis

By Huaiping Ming, Lei He, Frank K. Soong. 2020

5 citations recorded by the source corpus.

Neural end-to-end text-to-speech (TTS) , which adopts either a recurrent model, e.g. Tacotron, or an attention one, e.g. Transformer, to characterize a speech utterance, has achieved significant improvement of speech synthesis. However, it is still very challenging to deal with different sentence lengths, particularly, for long sentences where sequence model has limitation of the effective context length. We propose a novel segment-Transformer (s-Transformer), which models speech at segment level where recurrence is reused via cached memories for both the encoder and decoder. Long-range contexts can be captured by the extended memory, meanwhile, the encoder-decoder attention on segment which is much easier to handle. In addition, we employ a modified relative positional self attention to generalize sequence length beyond a period possibly unseen in the training data. By comparing the proposed s-Transformer with the standard Transformer, on short sentences, both achieve the same MOS scores of 4.29, which is very close to 4.32 by the recordings; similar scores of 4.22 vs 4.2 on long sentences, and significantly better for extra-long sentences with a gain of 0.2 in MOS. Since the cached memory is updated with time, the s-Transformer generates rather natural and coherent speech for a long period of time.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Xu Tan
  2. Tao Qin
  3. Tie-Yan Liu
  4. Frank K. Soong
  5. Sheng Zhao
  6. Yi Ren
  7. Zhou Zhao
  8. Lei He

In Signal Processing

  1. Xu Tan
  2. Tie-Yan Liu
  3. Tao Qin
  4. Frank K. Soong
  5. Sheng Zhao
  6. Yi Ren
  7. Zhou Zhao
  8. Lei He

Citations in the corpus