Syntactically Guided Neural Machine Translation

By Felix Stahlberg, Eva Hasler, Aurelien Waite, Bill Byrne. 2016

66 citations recorded by the source corpus.

We investigate the use of hierarchical phrase-based SMT lattices in end-to-end neural machine translation (NMT). Weight pushing transforms the Hiero scores for complete translation hypotheses, with the full translation grammar score and full n-gram language model score, into posteriors compatible with NMT predictive probabilities. With a slightly modified NMT beam-search decoder we find gains over both Hiero and NMT decoding alone, with practical advantages in extending NMT to very large input and output vocabularies.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Graph Convolutional Encoders for Syntax-aware Neural Machine Translation (2017, methodology): “Stahlberg et al. (2016) use a pruned lattice from a hierarchical phrase-based model (hiero) to constrain NMT. Hiero trees are not syntactically-aware, but instead constrained by symmetrized word alignments.”
Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem (2020, positive): “An alternative approach for avoiding catastrophic forgetting takes inspiration from lattice rescoring for NMT (Stahlberg et al., 2016) and Grammatical Error Correction (Stahlberg et al., 2019).”
Neural Machine Translation with Reconstruction (2016, background): “Several researchers rescore word candidates with additional features, such as language model probability (Gulcehre et al. 2015) and SMT features (He et al. 2016; Stahlberg et al. 2016).”
Variable length word encodings for neural translation models (2016, methodology): “The contribution of this dissertation is two-fold: • We implement a target side RNN language model based on Zaremba et al. (2014) using TensorFlow (Abadi et al. (2016)) in the framework established by Stahlberg et al. (2016), and thereby allowing lattice rescoring in conjunction with Hiero scores.”
First the Worst: Finding Better Gender Translations During Beam Search (2021, methodology): “We can now perform constrained beam search, which can encourage NMT to output specific vocabulary (Stahlberg et al., 2016; Khayrallah et al., 2017).”
The University of Cambridge’s Machine Translation Systems for WMT18 (2018, background): “Therefore, the literature about hybrid NMT-SMT systems is also vast, ranging from rescoring and reranking methods (Neubig et al., 2015; Stahlberg et al., 2016; Khayrallah et al., 2017; Grund-kiewicz and Junczys-Dowmunt, 2018; Avramidis et al., 2016; Marie and Fujita, 2018), MBR-based formalisms…”
The Edit Distance Transducer in Action: The University of Cambridge English-German System at WMT16 (2016, background): “, 2015) are very different and have complementary strengths and weaknesses (Neubig et al., 2015; Stahlberg et al., 2016).”
Neural Machine Translation: A Review (2019, unknown): “+ Produces richer output lattices [235].”
Neural Machine Translation with Source-Side Latent Graph Parsing (2017, background): “Most of the existing NMT models treat each sentence as a sequence of tokens, but recent studies suggest that syntactic information can help improve translation accuracy (Eriguchi et al., 2016b, 2017; Sennrich and Haddow, 2016; Stahlberg et al., 2016).”
Learning to Parse and Translate Improves Neural Machine Translation (2017, methodology): “Stahlberg et al. (2016) let the lattice from a hierarchical phrase-based system guide the decoding process of neural machine translation, which results in two separate models rather than a single end-to-end one.”
Neural Machine Translation With Explicit Phrase Alignment (2019, unknown): “A key difference of our approach from conventional phrase-based SMT (Koehn et al., 2003) is that unaligned source and target phrases are allowed to reduce the discrepancy between the search spaces of SMT and NMT models.”
Towards String-To-Tree Neural Machine Translation (2017, methodology): “…learning: their system included a single encoder with multiple decoders, one of which attempts to predict the parse-tree of the source sentence; Stahlberg et al. (2016) proposed a hybrid approach in which translations are scored by combining scores from an NMT system with scores from a Hiero…”
Syntax-Guided Controlled Generation of Paraphrases (2020, background): “Incorporating complex syntactic information has shown promising results in neural machine translation (Stahlberg et al., 2016; Aharoni and Goldberg, 2017; Yang et al., 2019), data-to-text generation (Peng et al., 2019), abstractive text-summarization (Cao et al., 2018) and adversarial text…”
On the Linguistic Representational Power of Neural Machine Translation Models (2019, background): “There have also been recent attempts to integrate syntax into NMT (Eriguchi, Hashimoto, and Tsuruoka 2016; Stahlberg et al. 2016; Aharoni and Goldberg 2017; Chen et al. 2017; Wu et al. 2017), but sequence-to-sequence NMT models without explicit syntax are the state of the art at the moment (Pinnis…”
Chunk-Based Bi-Scale Decoder for Neural Machine Translation (2017, methodology): “, 2008), and a lot of previous work explore to incorporate the syntactic information in NMT, which shows the effectiveness of the syntactic information (Stahlberg et al., 2016).”