Assessing the Tolerance of Neural Machine Translation Systems Against Speech Recognition Errors

By Nicholas Ruiz, Mattia Antonino Di Gangi, Nicola Bertoldi, Marcello Federico. 2017

23 citations recorded by the source corpus.

Machine translation systems are conventionally trained on textual resources that do not model phenomena that occur in spoken language. While the evaluation of neural machine translation systems on textual inputs is actively researched in the literature , little has been discovered about the complexities of translating spoken language data with neural models. We introduce and motivate interesting problems one faces when considering the translation of automatic speech recognition (ASR) outputs on neural machine translation (NMT) systems. We test the robustness of sentence encoding approaches for NMT encoder-decoder modeling, focusing on word-based over byte-pair encoding. We compare the translation of utterances containing ASR errors in state-of-the-art NMT encoder-decoder systems against a strong phrase-based machine translation baseline in order to better understand which phenomena present in ASR outputs are better represented under the NMT framework than approaches that represent translation as a linear model.

Subfields: Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Robust Neural Machine Translation with Joint Textual and Phonetic Embedding (2018, background): “One example is speech translation (Ruiz et al., 2017; Ruiz and Federico, 2015; Ma et al., 2018), where an ASR system may output correct or almost correct phoneme sequences, but transcribe some words into their homophones.”
Enhancing Transformer for End-to-end Speech-to-Text Translation (2019, background): “Error propagation is particularly problematic for the SLT task (Ruiz et al., 2017), in which MT would be signiﬁcantly penalized by errors resulting from the previous ASR processing step.”
Data Augmentation for Pipeline-Based Speech Translation (2020, background): “However, since ASR systems can introduce errors that a standard MT system has not seen during training and thus cannot handle, the translation quality can suffer, even to an extent where the translations are incomprehensible [7].”
Toward Robust Neural Machine Translation for Noisy Input Sequences (2017, background): “The now prevalent neural sequence-to-sequence models [1, 2, 3] have been identiﬁed to be especially sensitive to noisy data [4, 5, 6], and more speciﬁcally to corrupted inputs due to erroneous ASR [7].”
Neural Machine Translation: A Review (2019, background): “naturally occurring noise at training (Khayrallah & Koehn, 2018) and test (Belinkov & Bisk, 2017; Cheng et al., 2018; Michel & Neubig, 2018; Ruiz et al., 2017) time.”