Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matching

By Chih-Kuan Yeh, Jianshu Chen, Chengzhu Yu, Dong Yu. 2018

42 citations recorded by the source corpus.

We consider the problem of training speech recognition systems without using any labeled data, under the assumption that the learner can only access to the input utterances and a phoneme language model estimated from a non-overlapping corpus. We propose a fully unsupervised learning algorithm that alternates between solving two sub-problems: (i) learn a phoneme classifier for a given set of phoneme segmentation boundaries, and (ii) refining the phoneme boundaries based on a given classifier. To solve the first sub-problem, we introduce a novel unsupervised cost function named Segmental Empirical Output Distribution Matching, which generalizes the work in (Liu et al., 2017) to segmental structures. For the second sub-problem, we develop an approximate MAP approach to refining the boundaries obtained from Wang et al. (2017). Experimental results on TIMIT dataset demonstrate the success of this fully unsupervised phoneme recognition system, which achieves a phone error rate (PER) of 41.6%. Although it is still far away from the state-of-the-art supervised systems, we show that with oracle boundaries and matching language model, the PER could be improved to 32.5%.This performance approaches the supervised system of the same model architecture, demonstrating the great potential of the proposed method.

Subfields: Artificial Intelligence, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Tao Qin
2. Tie-Yan Liu
3. Xu Tan
4. Hung-yi Lee
5. Wei-Ning Hsu
6. James R. Glass
7. Zhou Zhao
8. Abdel-rahman Mohamed

In Speech & Audio

1. Wei-Ning Hsu
2. Hung-yi Lee
3. Xu Tan
4. Yi Ren
5. Abdel-rahman Mohamed
6. Zhou Zhao
7. Jinyu Li
8. Tao Qin

In Signal Processing

1. Hung-yi Lee
2. Xu Tan
3. Wei-Ning Hsu
4. Tie-Yan Liu
5. Tao Qin
6. Abdel-rahman Mohamed
7. Yi Ren
8. Zhou Zhao

Citations in the corpus

• Unsupervised Speech Recognition (2021, methodology): “3 Unsupervised learning - unmatched setup EODM (Yeh et al., 2019) 5-gram - 41.”
• Almost Unsupervised Text to Speech and Automatic Speech Recognition (2019, methodology): “Therefore, a variety of techniques for low-resource and zero-resource ASR and TTS have been proposed recently, including un-supervised ASR (Yeh et al., 2019; Chen et al., 2018a; Liu et al., 2018; Chen et al., 2018b), low-resource ASR (Chuang-suwanich, 2016; Dalmia et al., 2018; Zhou et al., 2018),…”
• LRSpeech: Extremely Low-Resource Speech Synthesis and Recognition (2020, background): “ens of hours × × unpaired speech (multi-speaker, low-quality) ✓ ✓ dozens of hours ✓ unpaired text ✓ ✓ ✓ ✓ Related Work TTS [22, 28, 30, 35] [2, 12, 23, 31] Our Work / ASR [6, 10, 11] [16, 32, 33, 39] [8, 24, 45] Table 1: The data resource to build TTS and ASR systems and the corresponding related wo”
• Self-Supervised Speech Representation Learning: A Review (2022, methodology): “Other ways to map speech and text include via segmental empirical output distribution matching (segmental empirical-ODM) [290] and decipherment algorithm [294].”