A study of multilingual speech recognition

By Fuliang Weng, Harry Bratt, Leonardo Neumeyer, Andreas Stolcke. 1997

This paper describes our work in developing multilingual (Swedish and English) speech recognition systems in the ATIS domain. The acoustic component of the multilingual systems is realized through sharing Gaussian codebooks across Swedish and English allophones. The language model (LM) components are constructed by training a statistical bigram model, with a common backoff node, on bilingual texts, and by combining two monolingual LMs into a probabilistic finite state grammar. This system uses a single decoder for Swedish and English sentences, and is capable of recognizing sentences with words from both languages. Preliminary experiments show that sharing acoustic models across the two languages has not resulted in improved performance, while sharing a backoff node at the LM component provides flexibility and ease in recognizing bilingual sentences at the expense of a slight increase in word error rate in some cases. As a by-product, the bilingual decoder also achieves good performance on language identification (LID).

Subfields: Speech & Audio

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Speech & Audio

Citations in the corpus

Polyglot Neural Language Models: A Case Study in Cross-Lingual Phonetic Representation Learning (2016, background): “Interpolation of monolingual LMs is an alternative to obtain a multilingual model (Harbeck et al., 1997; Weng et al., 1997).”
Automatic speech recognition and speech variability: A review (2007, background): “The mapping between phoneme models of different languages can be derived from data (Weng et al., 1997) or determined from phonetic knowledge (Uebler, 2001), but this is far from obvious as each language has his own characteristic set of phonetic units and associated distinctive features.”
Current trends in multilingual speech processing (2011, methodology): “Research in the direction of building multilingual language models has mainly focussed on (Weng et al 1997; Fügen et al 2003; Khudanpur 2006) (i) estimation of a single language model using the data from all languages, or (ii) estimation of a language model for each language separately including…”