Audio Word2vec: Sequence-to-Sequence Autoencoding for Unsupervised Learning of Audio Segmentation and Representation

By Yi-Chen Chen, Sung-Feng Huang, Hung-yi Lee, Yu-Hsuan Wang, Chia-Hao Shen. 2019

38 citations recorded by the source corpus.

In text, word2vec transforms each word into a fixed-size vector used as the basic component in applications of natural language processing. Given a large collection of unannotated audio, audio word2vec can also be trained in an unsupervised way using a sequence-to-sequence autoencoder (SA). These vector representations are shown to effectively describe the sequential phonetic structures of the audio segments. In this paper, we further extend this research in the following two directions. First, we disentangle phonetic information and speaker information from the SA vector representations. Second, we extend audio word2vec from the word level to the utterance level by proposing a new segmental audio word2vec in which unsupervised spoken word boundary segmentation and audio word2vec are jointly learned and mutually enhanced, and utterances are directly represented as sequences of vectors carrying phonetic information. This is achieved by means of a segmental sequence-to-sequence autoencoder, in which a segmentation gate trained with reinforcement learning is inserted in the encoder.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Shujie Liu
3. Zhuo Chen
4. Yu Wu
5. Takuya Yoshioka
6. Sanyuan Chen
7. Dong Yu
8. Furu Wei

In Signal Processing

1. Jinyu Li
2. Zhuo Chen
3. Takuya Yoshioka
4. Shujie Liu
5. Yu Wu
6. Sanyuan Chen
7. Dong Yu
8. Naoyuki Kanda

Citations in the corpus

• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing (2021, background): “The research line of generative learning can be traced back to the auto-encoding model, which reconstructs the whole speech from latent variables, either continuous [25]– [27] or discrete [28].”
• UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Data (2021, methodology): “Chen et al. (2019) use an autoencoder to perform full reconstruction.”
• End-to-End Speech Recognition: A Survey (2023, unknown): “Pretraining approaches utilizing untranscribed audio, large-scale semi-supervised data and/or multilingual data [120], [121], [122], [123], [124], [125], [126], [127], [128], [129], [130], [131] would deserve”