Learning to Parse and Translate Improves Neural Machine Translation

By Akiko Eriguchi, Yoshimasa Tsuruoka, Kyunghyun Cho. 2017

153 citations recorded by the source corpus.

There has been relatively little attention to incorporating linguistic prior to neural machine translation. Much of the previous work was further constrained to considering linguistic prior on the source side. In this paper, we propose a hybrid model, called NMT+RNNG, that learns to parse and translate by combining the recurrent neural network grammar into the attention-based neural machine translation. Our approach encourages the neural machine translation model to incorporate linguistic prior during training, and lets it translate on its own afterward. Extensive experiments with four language pairs show the effectiveness of the proposed NMT+RNNG.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Graph Convolutional Encoders for Syntax-aware Neural Machine Translation (2017, background): “Eriguchi et al. (2017) multi-task with a target-side RNNG parser (Dyer et al., 2016) and improve on various language pairs with English on the target side.”
Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling (2017, background): “Beyond SRL, there have been many proposals on how to incorporate syntactic information in RNN models, for example, in the context of neural machine translation (Eriguchi et al., 2017; Sennrich and Haddow, 2016).”
Deep Learning for Symbolic Mathematics (2019, methodology): “Alternative approaches have been proposed to generate trees, such as Tree-LSTM (Tai et al., 2015) or Recurrent Neural Network Grammars (RNNG) (Dyer et al., 2016; Eriguchi et al., 2017).”
Transforming Question Answering Datasets Into Natural Language Inference Datasets (2018, background): “…2017; Vaswani et al., 2017), we believe it should be possible to further increase the data efﬁciency and performance, especially by leveraging models that incorporate syntactic structure (Chen et al., 2017; Eriguchi et al., 2017) or a more transducer-like structure (Graves, 2012; Yu et al., 2016).”
Search Engine Guided Non-Parametric Neural Machine Translation (2017, background): “…et al., 2015; Dong et al., 2015), multi-source translation (Zoph and Knight, 2016; Firat et al., 2016b; Garmash and Monz, 2016), multimodal translation (Caglayan et al., 2016) and linguistic knowledge guided translation (Nadejde et al., 2017; Aharoni and Goldberg, 2017; Eriguchi et al., 2017).”
Joint Parsing and Generation for Abstractive Summarization (2019, methodology): “Incorporating syntax is found to improve translation (Li et al. 2017a; Eriguchi, Tsuruoka, and Cho 2017; Wu et al. 2017; Wang et al. 2018).”
Graph Convolutional Networks With Argument-Aware Pooling for Event Detection (2018, background): “…various natural language processing tasks, including sentiment analysis (Socher et al. 2013; Mou et al. 2015), dependency parsing (Le and Zuidema 2014; Dyer et al. 2015), relation extraction (Li et al. 2015b; Nguyen and Grishman 2016d), machine translation (Eriguchi, Tsuruoka, and Cho 2017) etc.”
Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models (2021, background): “…2018; Devlin et al., 2019), multilingual extensions of pre-trained encoders have been proven effective in learning deep contextual cross-lingual representations (Eriguchi et al., 2017; Lample and Conneau, 2019; Wu and Dredze, 2019b; Siddhant et al., 2020; Pires et al., 2019; Pfeiffer et al., 2020).”
Predicting Target Language CCG Supertags Improves Neural Machine Translation (2017, unknown): “Concurrently with our work, Aharoni and Goldberg (2017) proposed serializing the target constituency trees and Eriguchi et al. (2017) model target dependency relations by augmenting the NMT decoder with a RNN grammar (Dyer et al., 2016).”
Graph-to-Sequence Learning using Gated Graph Neural Networks (2018, background): “Other approaches have investigated the use of syntax in the target language (Aharoni and Goldberg, 2017; Eriguchi et al., 2017).”
Unsupervised Recurrent Neural Network Grammars (2019, background): “, 2016) and conditional (Yin and Neubig, 2017; Alvarez-Melis and Jaakkola, 2017; Rabinovich et al., 2017; Aharoni and Goldberg, 2017; Eriguchi et al., 2017; Wang et al., 2018; Gu et al., 2018) cases.”
Novel positional encodings to enable tree-based transformers (2019, methodology): “Sequence-to-sequence models without explicit tree modeling have been applied to tree generation using only a simple linearization of the tree structure (Vinyals et al., 2015; Eriguchi et al., 2017; Aharoni & Goldberg, 2017).”
Multitask Learning with Low-Level Auxiliary Tasks for Encoder-Decoder Based Speech Recognition (2017, background): “Its application to deep neural networks has been successful in a variety of settings in speech and language processing [22, 23, 24, 25, 26, 27, 28].”
Non-Monotonic Sequential Text Generation (2019, methodology): “In investigating this question, we are solely interested in considering non-monotonic generation that does not rely on external supervision, such as parse trees (Eriguchi et al., 2017; Aharoni & Goldberg, 2017).”
Scheduled Multi-Task Learning: From Syntax to Translation (2018, background): “2016; Kuncoro et al., 2017; Niehues and Cho, 2017; Sennrich and Haddow, 2016; Eriguchi et al., 2017; Aharoni and Goldberg, 2017; Nadejde et al., 2017; Bastings et al., 2017; Matthews et al., 2018) has proven to be beneficial, achieving higher results in language modeling and machine translation.”
Multi-Task Learning for Sequence Tagging: An Empirical Study (2018, methodology): “…MWE (Constant and Nivre, 2016), or with semantic role labeling (SRL) (Shi et al., 2016); machine translation (TRANSLATE) with POS or DEP (Niehues and Cho, 2017; Eriguchi et al., 2017); sentence extraction and COM (Martins and Smith, 2009; Berg-Kirkpatrick et al., 2011; Almeida and Martins, 2013).”
Softmax Tempering for Training Neural Machine Translation Models (2020, background): “…sometimes be considered as data regularization and comes in the form of monolingual or cross-lingual (multilingual) transfer learning (Zoph et al., 2016; Song et al., 2019), pseudo-parallel data generation (back-translation) (Sennrich et al., 2016), or multi-task learning (Eriguchi et al., 2017).”