Tree-to-Sequence Attentional Neural Machine Translation

By Akiko Eriguchi, Kazuma Hashimoto, Yoshimasa Tsuruoka. 2016

279 citations recorded by the source corpus.

Most of the existing Neural Machine Translation (NMT) models focus on the conversion of sequential data and do not directly use syntactic information. We propose a novel end-to-end syntactic NMT model, extending a sequence-to-sequence model with the source-side phrase structure. Our model has an attention mechanism that enables the decoder to generate a translated word while softly aligning it with phrases as well as words of the source sentence. Experimental results on the WAT'15 English-to-Japanese dataset demonstrate that our proposed model considerably outperforms sequence-to-sequence attentional NMT models and compares favorably with the state-of-the-art tree-to-string SMT system.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Graph Convolutional Encoders for Syntax-aware Neural Machine Translation (2017, methodology): “Eriguchi et al. (2016) parse English sentences with an HPSG parser and use a Tree-LSTM to encode the internal nodes of the tree.”
Tree-to-tree Neural Networks for Program Translation (2018, methodology): “We evaluate our tree-to-tree model against a sequence-to-sequence model [4, 31], a sequence-to-tree model [11], and a tree-to-sequence model [13].”
Investigating Failures of Automatic Translation in the Case of Unambiguous Gender (2021, background): “For example, Eriguchi et al. (2016) model source-side syntax while Aharoni and Goldberg (2017) proposed models that generate linearized dependency trees.”
Generating Sentences from Disentangled Syntactic and Semantic Spaces (2019, background): “Recently, researchers have shown that explicitly syntactic modeling improves the generation quality in sequence-to-sequence models (Eriguchi et al., 2016; Zhou et al., 2017; Li et al., 2017; Chen et al., 2017).”
Learning to Parse and Translate Improves Neural Machine Translation (2017, methodology): “Eriguchi et al. (2016) instead replaced the sequential encoder with a tree-based encoder which computes the representation of the source sentence following its parse tree.”
A Tree-to-Sequence Model for Neural NLG in Task-Oriented Dialog (2019, methodology): “Moryossef et al. (2019) add a symbolic text planning step where facts are grouped and ordered in the input; in contrast to our work though, their approach uses standard Seq2Seq models for realization and leaves no ordering choices to the model.”
A Tree-based Decoder for Neural Machine Translation (2018, background): “…improve NMT 1 Our code is available at https://github.com/cindyxinyiwang/TrDec_pytorch. quality, either through syntactic encoders (Li et al., 2017; Eriguchi et al., 2016), multi-task learning objectives (Chen et al., 2017; Eriguchi et al., 2017), or direct addition of syntactic tokens to the…”
Neural Machine Translation with Source-Side Latent Graph Parsing (2017, methodology): “In particular, Eriguchi et al. (2016b) introduced a tree-to-sequence NMT model by building a tree-structured encoder on top of a standard sequential encoder, which motivated the use of the dependency composition vectors in our proposed model.”
Chunk-based Decoder for Neural Machine Translation (2017, methodology): “This simple framework allows researchers to incorporate the structure of the source sentence as in SMT by leveraging various architectures as the encoder (Kalchbrenner and Blunsom, 2013; Sutskever et al., 2014; Cho et al., 2014b; Eriguchi et al., 2016b).”
Tree-Transformer: A Transformer-Based Method for Correction of Tree-Structured Data (2019, unknown): “Several types of tree encoders exist [9, 10, 11, 12].”
Does Neural Machine Translation Benefit from Larger Context? (2017, methodology): “We propose a novel larger-context neural machine translation model based on the recent works on larger-context language modelling (Wang and Cho, 2016) and multi-way, multilingual neural machine translation (Firat et al., 2016).”
Structural Neural Encoders for AMR-to-text Generation (2019, methodology): “This was shown to be effective for other tasks with both TreeLSTMs (Eriguchi et al., 2016; Chen et al., 2017) and GCNs (Marcheggiani and Titov, 2017; Cetoli et al.”
Program Language Translation Using a Grammar-Driven Tree-to-Tree Model (2018, background): “els, which map input sequences to output sequences, have achieved great performance (Bahdanau et al., 2014; Cho et al., 2014; Eriguchi et al., 2016; Vaswani et al., 2017).”
Neural Machine Translation (2016, unknown): “convolution network as encoder [Kalchbrenner and Blunsom, 2013] TreeLSTM as encoder [Eriguchi et al., 2016] modifications to attention mechanism [Luong et al.”
Predicting Target Language CCG Supertags Improves Neural Machine Translation (2017, methodology): “…and Blunsom (2013); Cho et al. (2014a) capture the hierarchical aspects of language implicitly by using convolutional neural networks, while Eriguchi et al. (2016) use the parse tree of the source sentence to guide the recurrence and attention model in tree-to-sequence NMT. Luong et al.…”