Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder

By Huadong Chen, Shujian Huang, David Chiang, Jiajun Chen. 2017

152 citations recorded by the source corpus.

Most neural machine translation (NMT) models are based on the sequential encoder-decoder framework, which makes no use of syntactic information. In this paper, we improve this model by explicitly incorporating source-side syntactic trees. More specifically, we propose (1) a bidirectional tree encoder which learns both sequential and tree structured representations; (2) a tree-coverage model that lets the attention depend on the source-side syntax. Experiments on Chinese-English translation demonstrate that our proposed models outperform the sequential attentional model as well as a stronger baseline with a bottom-up tree encoder and word coverage.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Towards Open Intent Discovery for Conversational Text (2019, background): “Moreover, adding linguistic structure to existing models has been shown to improve their performance across a wide range of NLU tasks, such as word embedding [38], machine translation [4], named entity recognition [21], and semantic role labelling [16].”
Transforming Question Answering Datasets Into Natural Language Inference Datasets (2018, background): “…2017; Vaswani et al., 2017), we believe it should be possible to further increase the data efﬁciency and performance, especially by leveraging models that incorporate syntactic structure (Chen et al., 2017; Eriguchi et al., 2017) or a more transducer-like structure (Graves, 2012; Yu et al., 2016).”
Improved Neural Machine Translation with Chinese Phonologic Features (2018, methodology): “done to incorporate linguistic knowledge into NMT models [4,6,12,23].”
Improved Code Summarization via a Graph Neural Network (2020, background): “ese relationships could be the hidden structures of a sentence, the parts of speech [8], dependency parsing trees [59], or the sequence of tokens [7].”
Structural Neural Encoders for AMR-to-text Generation (2019, methodology): “In order to use information from both incoming and outgoing edges (parents and children), we employ bidirectional TreeLSTMs (Eriguchi et al., 2016), where the bottom-up pass is followed by a top-down pass.”
Neural Models for Reasoning over Multiple Mentions Using Coreference (2018, methodology): “Recent work has used syntax, in the form of dependency trees, to replace the sequential recency bias in RNNs with a syntactic recency bias (Tai et al., 2015; Swayamdipta, 2017; Qian et al., 2017; Chen et al., 2017).”
Neural Machine Translation with Source-Side Latent Graph Parsing (2017, background): “…al., 2015; Sutskever et al., 2014), researchers have recently started investigating into the use of syntactic structures in NMT models (Bastings et al., 2017; Chen et al., 2017; Eriguchi et al., 2016a,b, 2017; Li et al., 2017; Sennrich and Haddow, 2016; Stahlberg et al., 2016; Yang et al., 2017).”
A Tree-based Decoder for Neural Machine Translation (2018, background): “…quality, either through syntactic encoders (Li et al., 2017; Eriguchi et al., 2016), multi-task learning objectives (Chen et al., 2017; Eriguchi et al., 2017), or direct addition of syntactic tokens to the target sequence (Nadejde et al., 2017; Aharoni and Goldberg,…”
Generating Sentences from Disentangled Syntactic and Semantic Spaces (2019, background): “Recently, researchers have shown that explicitly syntactic modeling improves the generation quality in sequence-to-sequence models (Eriguchi et al., 2016; Zhou et al., 2017; Li et al., 2017; Chen et al., 2017).”
RNNs Implicitly Implement Tensor Product Representations (2018, methodology): “Our ﬁnal topology is tree-based RNNs (Pollack, 1990; Socher et al., 2010), speciﬁcally the Tree-GRU encoder of Chen et al. (2017) and the tree decoder of Chen et al. (2018).”