Overcoming the Curse of Sentence Length for Neural Machine Translation using Automatic Segmentation

By Jean Pouget-Abadie, Dzmitry Bahdanau, Bart van Merrienboer, Kyunghyun Cho, Yoshua Bengio. 2014

84 citations recorded by the source corpus.

The authors of (Cho et al., 2014a) have shown that the recently introduced neural network translation systems suffer from a significant drop in translation quality when translating long sentences, unlike existing phrase-based translation systems. In this paper, we propose a way to address this issue by automatically segmenting an input sentence into phrases that can be easily translated by the neural network translation model. Once each segment has been independently translated by the neural machine translation model, the translated clauses are concatenated to form a final translation. Empirical results show a significant improvement in translation quality for long sentences.

Subfields: Artificial Intelligence, Language and Linguistics, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Kyunghyun Cho
2. Graham Neubig
3. Yonatan Belinkov
4. Mohit Bansal
5. Yoshua Bengio
6. Felix Stahlberg
7. Philipp Koehn
8. Jianfeng Gao

In Language and Linguistics

1. Marcello Federico
2. Oriol Vinyals
3. Yoshua Bengio
4. Philipp Koehn
5. Kyunghyun Cho
6. Quoc V. Le
7. Mauro Cettolo

In Natural Language Processing

1. Kyunghyun Cho
2. Graham Neubig
3. Yonatan Belinkov
4. Felix Stahlberg
5. Yoshua Bengio
6. Christopher D. Manning
7. Philipp Koehn
8. Mohit Bansal

Citations in the corpus

• Sequence to Sequence Learning with Neural Networks (2014, background): “We were initially convinced that the LSTM would fail on long sentences due to its limited memory, and other researchers reported poor performance on long sentences with a model similar to ours [5, 2, 26].”
• Neural Machine Translation by Jointly Learning to Align and Translate (2014, methodology): “We conjectured that the use of a fixed-length context vector is problematic for translating long sentences, based on a recent empirical study reported by Cho et al. (2014b) and Pouget-Abadie et al. (2014).”
• Six Challenges for Neural Machine Translation (2017, background): “A well-known flaw of early encoder-decoder NMT models was the inability to properly translate long sentences (Cho et al., 2014; Pouget-Abadie et al., 2014).”
• Neural versus Phrase-Based Machine Translation Quality: a Case Study (2016, background): “Following previous work (Cho et al., 2014a; Pouget-Abadie et al., 2014; Bahdanau et al., 2015; Luong et al., 2015), we investigate how sentence length affects overall translation quality.”
• Length bias in Encoder Decoder Models and a Case for Global Conditioning (2016, methodology): “Later more powerful encoders based on attention achieved greater accuracy (Bahdanau et al., 2014) on long sequences.”