Learning Word Embeddings from Speech

By Yu-An Chung, James R. Glass. 2017

19 citations recorded by the source corpus.

In this paper, we propose a novel deep neural network architecture, Sequence-to-Sequence Audio2Vec, for unsupervised learning of fixed-length vector representations of audio segments excised from a speech corpus, where the vectors contain semantic information pertaining to the segments, and are close to other vectors in the embedding space if their corresponding segments are semantically similar. The design of the proposed model is based on the RNN Encoder-Decoder framework, and borrows the methodology of continuous skip-grams for training. The learned vector representations are evaluated on 13 widely used word similarity benchmarks, and achieved competitive results to that of GloVe. The biggest advantage of the proposed model is its capability of extracting semantic information of audio segments taken directly from raw speech, without relying on any other modalities such as text or images, which are challenging and expensive to collect and annotate.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Unsupervised Speech Representation Learning Using WaveNet Autoencoders (2019, neutral): “[76] replace the segmental autoencoder with a model that instead predicts a nearby speech segment and demonstrate that the representation shares many properties with word embeddings.”
Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces (2018, background): “Recently, there is an increasing interest in learning the semantics of a language directly, and only from raw speech [24, 27, 28].”
A Brief Overview of Unsupervised Neural Speech Representation Learning (2022, methodology): “Inspired by the work on semantic word embeddings for text (Mikolov et al. 2013), the sequence-to-sequence framework has also been used to implement speech-based versions of the skipgram and continuous bag-of-words models (Chung and Glass 2017, 2018).”