Phonetic posteriorgrams for many-to-one voice conversion without parallel data training

By Lifa Sun, Kun Li, Hao Wang, Shiyin Kang, Helen Mei-Ling Meng. IEEE International Conference on Multimedia and Expo, 2016

341 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Signal Processing

View Paper: https://www.semanticscholar.org/paper/dc85adfda51c32c45beddc3b84a0c2c2ac9ae74e

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Junichi Yamagishi
2. Haizhou Li
3. Tao Qin
4. Dong Yu
5. Xu Tan
6. Lior Wolf
7. Tie-Yan Liu
8. Berrak Sisman

In Signal Processing

1. Junichi Yamagishi
2. Haizhou Li
3. Lior Wolf
4. Nima Mesgarani
5. Berrak Sisman
6. Dong Yu
7. Simon King
8. Zhenhua Ling

Citations in the corpus

• Any-to-Many Voice Conversion With Location-Relative Sequence-to-Sequence Modeling (2020, background): “Among these approaches, the one based on phonetic posteriorgrams (PPGs) is widely used [10]–[12].”
• WaveNet Vocoder with Limited Training Data for Voice Conversion (2018, methodology): “In this paper, the conversion model is built based on a framework similar to the context posterior probabilities (CPPs) based VC method [26].”
• Code-Switched Speech Synthesis Using Bilingual Phonetic Posteriorgram with Only Monolingual Corpora (2020, methodology): “Benefiting from its speaker-independent property, PPGs have been successfully used in voice conversion (VC) tasks [12–14].”
• Towards Natural Bilingual and Code-Switched Speech Synthesis Based on Mix of Monolingual Recordings and Cross-Lingual Voice Conversion (2020, methodology): “Different from other cross-lingual VC methods [16, 17, 18, 19, 20, 21, 22], we adapt state-of-the-art TTS model – Tacotron2 [1] to this cross-lingual VC task.”
• An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning (2020, methodology): “Phonetic Posteriograms, or PPG-based [32], approach represents”
• Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representations (2021, methodology): “One of the most popular text-based approaches is to use a pre-trained automatic speech recognition (ASR) network to extract a phonetic posteriogram (PPG) and use it as a linguistic feature [41].”
• LM-VC: Zero-Shot Voice Conversion via Speech Generation Based on Language Models (2023, methodology): “This collaboration is achieved through shallow fusion [21] with fusion weight λ : Note that shallow fusion is wildly used in ASR [29] to improve linguistic correctness during acoustic decoding.”
• Spoken Content and Voice Factorization for Few-Shot Speaker Adaptation (2020, background): “Inspired by the voice conversion task [13], which is primarily achieved by modifying spectral features while retaining the spoken content in the given speech signal, phone posteriorgram (PPG) [13] is usually used as the representation of spoken content and it is speaker independent.”
• NAUTILUS: A Versatile Voice Cloning System (2020, methodology): “Specifically, the PPG-based VC system [36] is essentially a TTS model stacked on top of an ASR model.”
• TTS Skins: Speaker Conversion via ASR (2019, methodology): “added, in order to improve the naturalness of the resulting audio. In these variational autoencoder works, spectral frames are used, while our WaveNet decoder uses the waveform. A recent line of work [13, 14, 15, 16] employs an ASR encoder to extract Phonetic Posterior Grams (PPGs), to perform voice”
• StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion (2021, methodology): “In VCC2020, only PGG-VC based methods [29] could achieve MOS higher than 4.”
• Adaptive Wavenet Vocoder for Residual Compensation in GAN-Based Voice Conversion (2018, background): “Recently, deep learning approaches such as restricted Boltzmann machine (RBM) [12, 13], long short-term memory (LSTM) [14, 15], and autoencoder [16, 17] are shown to be effective in modeling the relationship between source and target features more accurately than conventional techniques.”