Recurrent Neural Networks for Word Alignment Model

By Akihiro Tamura, Taro Watanabe, Eiichiro Sumita. 2014

64 citations recorded by the source corpus.

This study proposes a word alignment model based on a recurrent neural network (RNN), in which an unlimited alignment history is represented by recurrently connected hidden layers. We perform unsupervised learning using noise-contrastive estimation (Gutmann and Hyvarinen, 2010; Mnih and Teh, 2012), which utilizes artificially generated negative samples. Our alignment model is directional, similar to the generative IBM models (Brown et al., 1993). To overcome this limitation, we encourage agreement between the two directional models by introducing a penalty function that ensures word embedding consistency across two directional models during training. The RNN-based model outperforms the feed-forward neural network-based model (Yang et al., 2013) as well as the IBM Model 4 under Japanese-English and French-English word alignment tasks, and achieves comparable translation performance to those baselines for Japanese-English and Chinese-English translation tasks.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Word Alignment by Fine-tuning Embeddings on Parallel Corpora (2021, methodology): “Recently, there is a resurgence of interest in neural word alignment (Tamura et al., 2014; Alkhouli et al., 2018).”
SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings (2020, background): “…outperforms both statistical baselines. fast-Text becomes competitive for fewer than 1000 par-ENG-CES P r i o r W o r k (Dyer et al., 2011) .21 (Tamura et al., 2014) RNN .93 (¨Ostling, 2015a) .94 .06 .58 .42 .73 .27 (Legrand et al., 2016) .16 .10 (¨Ostling and Tiedemann, 2016) efmaral .08 .47…”