Attention-based Wav2Text with feature transfer learning

By Andros Tjandra, Sakriani Watiasri Sakti, Satoshi Nakamura. 2017

22 citations recorded by the source corpus.

Conventional automatic speech recognition (ASR) typically performs multi-level pattern recognition tasks that map the acoustic speech waveform into a hierarchy of speech units. But, it is widely known that information loss in the earlier stage can propagate through the later stages. After the resurgence of deep learning, interest has emerged in the possibility of developing a purely end-to-end ASR system from the raw waveform to the transcription without any predefined alignments and hand-engineered models. However, the successful attempts in end-to-end architecture still used spectral-based features, while the successful attempts in using raw waveform were still based on the hybrid deep neural network — Hidden Markov model (DNN-HMM) framework. In this paper, we construct the first end-to-end attention-based encoder-decoder model to process directly from raw speech waveform to the text transcription. We called the model as Attention-based Wav2Text. To assist the training process of the end-to-end model, we propose to utilize a feature transfer learning. Experimental results also reveal that the proposed Attention-based Wav2Text model directly with raw waveform could achieve a better result in comparison with the attentional encoder-decoder model trained on standard front-end filterbank features.

Subfields: Representation & Self-Supervised Learning, Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Representation & Self-Supervised Learning

In Speech & Audio

Citations in the corpus

Learning Filterbanks from Raw Speech for Phone Recognition (2017, background): “[8] explore an alternative architecture, with the intention to represent MFSC rather than gammatones.”
End-to-end Feedback Loss in Speech Chain Framework via Straight-through Estimator (2018, unknown): “17 Att MLP [21] 7.”
Data Augmentation for ASR Using TTS Via a Discrete Representation (2021, background): “Another direction that has been pursued is incorporation of unpaired text data into (part of) the ASR model training [8, 9, 10, 11, 12].”
Machine Speech Chain with One-shot Speaker Adaptation (2018, unknown): “…17.01 Att Enc-Dec [20] 17.68 Att Enc-Dec (ours) 17.35 Supervised training: WSJ train si284 (paired) → Upperbound Att Enc-Dec [19] 8.17 Att Enc-Dec [20] 7.69 Att Enc-Dec (ours) 7.12 Semi-supervised training: WSJ train si84 (paired) + train si200 (unpaired) Label propagation (greedy) 17.52 Label…”
Sequence-to-Sequence Asr Optimization Via Reinforcement Learning (2017, background): “This way, many complicated hand-engineered models can also be simplified by letting DNNs find their way to map from input to output spaces [5, 6, 7].”