Chunk-based Decoder for Neural Machine Translation

By Shonosuke Ishiwatari, Jingtao Yao, Shujie Liu, Mu Li, M. Zhou, Naoki Yoshinaga, Masaru Kitsuregawa, Weijia Jia. 2017

17 citations recorded by the source corpus.

Chunks (or phrases) once played a pivotal role in machine translation. By using a chunk rather than a word as the basic translation unit, local (intra-chunk) and global (inter-chunk) word orders and dependencies can be easily modeled. The chunk structure, despite its importance, has not been considered in the decoders used for neural machine translation (NMT). In this paper, we propose chunk-based decoders for (NMT), each of which consists of a chunk-level decoder and a word-level decoder. The chunk-level decoder models global dependencies while the word-level decoder decides the local word order in a chunk. To output a target sentence, the chunk-level decoder generates a chunk representation containing global information, which the word-level decoder then uses as a basis to predict the words inside the chunk. Experimental results show that our proposed decoders can significantly improve translation performance in a WAT ‘16 English-to-Japanese translation task.

Subfields: Artificial Intelligence, Molecular Biology, Model Architectures, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Takuya Yoshioka
2. Zhuo Chen
3. Yi Luo
4. Nima Mesgarani
5. Jinyu Li
6. Yossi Adi
7. John R. Hershey
8. Jonathan Le Roux

In Model Architectures

1. Yi Luo
2. Zhuo Chen
3. Takuya Yoshioka

Citations in the corpus

• Non-Autoregressive Neural Machine Translation (2017, neutral): “no significant difference in quality. There are also some work share a somehow simillar idea with our work: character-level NMT (Chung et al.,2016;Lee et al.,2016) and chunkbased NMT (Zhou et al.,2017;Ishiwatari et al., 2017). Unlike the SAT, these models are not able to produce multiple tokens (char”
• Dual-Path RNN: Efficient Long Sequence Modeling for Time-Domain Single-Channel Speech Separation (2019, methodology): “Compared with other approaches for arranging local and global RNN layers, or more general the hierarchical RNNs that perform sequence modeling in multiple time scales [14–19], the stacked DPRNN blocks iteratively and alternately perform the intra- and inter-chunk operations, which can be treated as ”