Phonetically-oriented word error alignment for speech recognition error analysis in speech translation
By Nicholas Ruiz, Marcello Federico. 2015
17 citations recorded by the source corpus.
We propose a variation to the commonly used Word Error Rate (WER) metric for speech recognition evaluation which incorporates the alignment of phonemes, in the absence of time boundary information. After computing the Levenshtein alignment on words in the reference and hypothesis transcripts, spans of adjacent errors are converted into phonemes with word and syllable boundaries and a phonetic Levenshtein alignment is performed. The phoneme alignment information is used to correct the word alignment labels in each error region. We demonstrate that our Phonetically-Oriented Word Error Rate (POWER) yields similar scores to WER with the added advantages of better word alignments and the ability to capture one-to-many alignments corresponding to homophonic errors in speech recognition hypotheses. These improved alignments allow us to better trace the impact of Levenshtein error types in speech recognition on downstream tasks such as speech translation.
Subfields: Artificial Intelligence, Natural Language Processing, Speech & Audio
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Marcello Federico
- Mattia Antonino Di Gangi
- Nicola Bertoldi
- Nicholas Ruiz
- Mingbo Ma
- Liang Huang
- Hao Xiong
- Hairong Liu
In Natural Language Processing
- Mattia Antonino Di Gangi
- Marcello Federico
- Nicola Bertoldi
- Nicholas Ruiz
- Zhongjun He
- Liang Huang
- Mingbo Ma
- Hao Xiong
In Speech & Audio
- Nicola Bertoldi
- Mattia Antonino Di Gangi
- Marcello Federico
- Nicholas Ruiz
- Liang Huang
- Mingbo Ma
- Juan Pino
- Changhan Wang
Citations in the corpus
- Robust Neural Machine Translation with Joint Textual and Phonetic Embedding (2018, background): “One example is speech translation (Ruiz et al., 2017; Ruiz and Federico, 2015; Ma et al., 2018), where an ASR system may output correct or almost correct phoneme sequences, but transcribe some words into their homophones.”
- Assessing the Tolerance of Neural Machine Translation Systems Against Speech Recognition Errors (2017, methodology): “Our experimental framework is similar to that of [2, 3], with the addition of a ranking experiment to evaluate the quality of NMT against our PBMT baseline.”