Chunk-Based Bi-Scale Decoder for Neural Machine Translation

By Hao Zhou, Zhaopeng Tu, Shujian Huang, Xiaohua Liu, Hang Li, Jiajun Chen. 2017

20 citations recorded by the source corpus.

In typical neural machine translation (NMT), the decoder generates a sentence word by word, packing all linguistic granularities in the same time-scale of RNN. In this paper, we propose a new type of decoder for NMT, which splits the decode state into two parts and updates them in two different time-scales. Specifically, we first predict a chunk time-scale state for phrasal modeling, on top of which multiple word time-scale states are generated. In this way, the target sentence is translated hierarchically from chunks to words, with information in different granularities being leveraged. Experiments show that our proposed model significantly improves the translation performance over the state-of-the-art NMT model.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Graham Neubig
2. Yang Liu
3. Mohit Bansal
4. M. Zhou
5. Jianfeng Gao
6. Felix Stahlberg
7. Furu Wei
8. Zhaopeng Tu

In Computer Vision and Pattern Recognition

1. Graham Neubig
2. Zhaopeng Tu
3. Hang Li
4. M. Zhou
5. Zhengdong Lu
6. Yang Liu
7. Shuming Shi
8. Lili Mou

In Machine Translation

1. Lei Li
2. Graham Neubig
3. Shujie Liu
4. Zhaopeng Tu
5. Naoki Yoshinaga
6. Yang Liu

Citations in the corpus

• Dual-Path RNN: Efficient Long Sequence Modeling for Time-Domain Single-Channel Speech Separation (2019, methodology): “Compared with other approaches for arranging local and global RNN layers, or more general the hierarchical RNNs that perform sequence modeling in multiple time scales [14–19], the stacked DPRNN blocks iteratively and alternately perform the intra- and inter-chunk operations, which can be treated as ”
• Neural Lattice Language Models (2018, methodology): “Many hierarchical neural language models have been proposed with a similar objective Koutnik et al. (2014; Zhou et al. (2017).”
• Coupling Distributed and Symbolic Execution for Natural Language Queries (2016, background): “, 2016) or chunking information (Zhou et al., 2017) in machine translation by coupling neural and external mechanisms.”
• Chunk-based Decoder for Neural Machine Translation (2017, background): “Finally, we noticed that (Zhou et al., 2017) (which is accepted at the same time as this paper) have also proposed a chunk-based decoder for NMT .”
• Learning to Remember Translation History with a Continuous Cache (2017, background): “…which means our model is required to train all the parameters and the other models cannot use mini-batch sorting, the training speeds for the models listed in Table 5 are 728.8, 159.8, 572.4, and 627.3, respectively. domain (Tu et al., 2016; Li et al., 2017; Zhou et al., 2017; Wang et al., 2017c).”
• Generating Sentences from Disentangled Syntactic and Semantic Spaces (2019, background): “Recently, researchers have shown that explicitly syntactic modeling improves the generation quality in sequence-to-sequence models (Eriguchi et al., 2016; Zhou et al., 2017; Li et al., 2017; Chen et al., 2017).”