A Tree-based Decoder for Neural Machine Translation

By Xinyi Wang, Hieu Pham, Pengcheng Yin, Graham Neubig. 2018

57 citations recorded by the source corpus.

Recent advances in Neural Machine Translation (NMT) show that adding syntactic information to NMT systems can improve the quality of their translations. Most existing work utilizes some specific types of linguistically-inspired tree structures, like constituency and dependency parse trees. This is often done via a standard RNN decoder that operates on a linearized target tree structure. However, it is an open question of what specific linguistic formalism, if any, is the best structural representation for NMT. In this paper, we (1) propose an NMT model that can naturally generate the topology of an arbitrary tree structure on the target side, and (2) experiment with various target tree structures. Our experiments show the surprising result that our model delivers the best improvements with balanced binary trees constructed without any linguistic knowledge; this model outperforms standard seq2seq models by up to 2.1 BLEU points, and other methods for incorporating target-side syntax by up to 0.7 BLEU.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Model Architectures

In Machine Translation

Citations in the corpus

Analysis Methods in Neural Language Processing: A Survey (2018, background): “Several challenge sets utilize existing test suites, either as a direct source of examples (Burchardt et al., 2017) or for searching similar naturally occurring examples (Wang et al., 2018a).”
Language learning using Speech to Image retrieval (2019, unknown): “[1, 2]).”
compare-mt: A Tool for Holistic Comparison of Language Generation Systems (2019, background): “4 and 5 of Wang et al. (2018) can be generated using sentence bucket analysis to measure “bucket=length, statistic=score” and “bucket=lengthdiff, statistic=count”.”
Unsupervised Recurrent Neural Network Grammars (2019, background): “…modeling, both for unconditional (Emami and Jelinek, 2005; Buys and Blunsom, 2015; Dyer et al., 2016) and conditional (Yin and Neubig, 2017; Alvarez-Melis and Jaakkola, 2017; Rabi-novich et al., 2017; Aharoni and Goldberg, 2017; Eriguchi et al., 2017; Wang et al., 2018; Gu et al., 2018) cases.”
Structural Information Preserving for Graph-to-Text Generation (2020, methodology): “Our work is remotely related to the previous efforts on string-to-tree neural machine translation (NMT) (Aharoni and Goldberg, 2017; Wu et al., 2017; Wang et al., 2018), which aims at generating target sentences with their syntactic trees.”
Joint Parsing and Generation for Abstractive Summarization (2019, methodology): “Incorporating syntax is found to improve translation (Li et al. 2017a; Eriguchi, Tsuruoka, and Cho 2017; Wu et al. 2017; Wang et al. 2018).”