Differentiable Sampling with Flexible Reference Word Order for Neural Machine Translation

By Weijia Xu, Xing Niu, Marine Carpuat. 2019

10 citations recorded by the source corpus.

Despite some empirical success at correcting exposure bias in machine translation, scheduled sampling algorithms suffer from a major drawback: they incorrectly assume that words in the reference translations and in sampled sequences are aligned at each time step. Our new differentiable sampling algorithm addresses this issue by optimizing the probability that the reference can be aligned with the sampled output, based on a soft alignment predicted by the model itself. As a result, the output distribution at each time step is evaluated with respect to the whole predicted sequence. Experiments on IWSLT translation tasks show that our approach improves BLEU compared to maximum likelihood and scheduled sampling baselines. In addition, our approach is simpler to train with no need for sampling schedule and yields models that achieve larger improvements with smaller beam sizes.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Pascale Fung
2. Andrea Madotto
3. Wenliang Dai
4. Nayeon Lee
5. Yan Xu
6. Etsuko Ishii
7. Ziwei Ji
8. Dan Su

In Natural Language Processing

1. Pascale Fung
2. Nayeon Lee
3. Andrea Madotto
4. Etsuko Ishii
5. Yejin Bang
6. Yan Xu
7. Rita Frieske
8. Dan Su

Citations in the corpus

• Survey of Hallucination in Natural Language Generation (2022, methodology): “[215] propose further improvement of the scheduled sampling algorithm for NMT by optimizing the probability of source and target word alignments.”