Neural Lattice-to-Sequence Models for Uncertain Inputs

By Matthias Sperber, Graham Neubig, Jan Niehues, A. Waibel. 2017

81 citations recorded by the source corpus.

The input to a neural sequence-to-sequence model is often determined by an up-stream system, e.g. a word segmenter, part of speech tagger, or speech recognizer. These up-stream models are potentially error-prone. Representing inputs through word lattices allows making this uncertainty explicit by capturing alternative sequences and their posterior probabilities in a compact form. In this work, we extend the TreeLSTM (Tai et al., 2015) into a LatticeLSTM that is able to consume word lattices, and can be used as encoder in an attentional encoder-decoder model. We integrate lattice posterior scores into this architecture by extending the TreeLSTM’s child-sum and forget gates and introducing a bias term into the attention mechanism. We experiment with speech translation lattices and report consistent improvements over baselines that translate either the 1-best hypothesis or the lattice without posterior scores.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates (2018, background): “Lattice-to-sequence models (Su et al., 2017; Sperber et al., 2017) are natural extension of sequence-to-sequence models, which represent inputs uncertainty through lattices.”
Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translation (2021, neutral): “…end-to-end speech translation (ST) is desired due to its advantages over the pipeline paradigm, such as low latency, alleviation of error propagation, and fewer parameters (Weiss et al., 2017; B´erard et al., 2018; Jia et al., 2019; Sperber et al., 2017; Zheng et al., 2020; Chen et al., 2021).”
Learning Shared Semantic Space for Speech-to-Text Translation (2021, unknown): “Cascaded ST The cascaded method is a more long-standing trend in ST (Sperber et al., 2017; Jan et al., 2018).”
Neural Lattice Language Models (2018, methodology): “This framework has been used before for calculating neural sentence representations involving lattices by Su et al. (2016) and Sperber et al.”
Neural Machine Translation: A Review (2019, background): “Lattices can represent uncertainty of upstream components such as speech recognizers [584] or tokenizers [585, 586].”
Transformer-Based Direct Speech-To-Speech Translation with Transcoder (2021, background): “Various studies have described how to use pre-trained speech recognition and machine translation to initialize speech translation models [5, 14, 15, 3].”
Mitigating Noisy Inputs for Question Answering (2019, methodology): “Machine translation [6] also uses ASR confidences; we evaluate similar methods on QA.”
Lattice-Based Transformer Encoder for Neural Machine Translation (2019, methodology): “As models mentioned above only use 1-best segmentation as inputs, lattice which can pack many different segmentations in a compact form has been widely used in statistical machine translation (SMT) (Xu et al., 2005; Dyer et al., 2008) and RNN-based NMT (Su et al., 2017; Sperber et al., 2017).”
Chinese NER Using Lattice LSTM (2018, methodology): “, 2017), as well as speech tokenization lattice (Sperber et al., 2017) and multi-granularity segmentation outputs (Su et al.”
Toward Robust Neural Machine Translation for Noisy Input Sequences (2017, background): “Finally, prior work has dealt with uncertain inputs from upstream components through explicit representation of the uncertainty, for example by directly translating word lattices produced by the speech recognizer [24, 25, 26, 27].”