Monotonic Infinite Lookback Attention for Simultaneous Machine Translation

By Naveen Arivazhagan, Colin Cherry, Wolfgang Macherey, Chung-Cheng Chiu, Semih Yavuz, Ruoming Pang, Wei Li, Colin Raffel. 2019

218 citations recorded by the source corpus.

Simultaneous machine translation begins to translate each source sentence before the source speaker is finished speaking, with applications to live and streaming scenarios. Simultaneous systems must carefully schedule their reading of the source sentence to balance quality against latency. We present the first simultaneous translation system to learn an adaptive schedule jointly with a neural machine translation (NMT) model that attends over all source tokens read thus far. We do so by introducing Monotonic Infinite Lookback (MILk) attention, which maintains both a hard, monotonic attention head to schedule the reading of the source sentence, and a soft attention head that extends from the monotonic head back to the beginning of the source. We show that MILk’s adaptive schedule allows it to arrive at latency-quality trade-offs that are favorable to those of a recently proposed wait-k strategy for many latency values.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Model Architectures, Machine Translation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Jinyu Li
  2. Yoshua Bengio
  3. Graham Neubig
  4. Quoc V. Le
  5. Yu Zhang
  6. Dong Yu
  7. James R. Glass
  8. Yonatan Belinkov

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Xiaogang Wang
  4. Ming-Hsuan Yang
  5. Trevor Darrell
  6. Jitendra Malik
  7. Alan L. Yuille
  8. Andrew Zisserman

In Model Architectures

  1. Jinyu Li
  2. Tara N. Sainath
  3. Shinji Watanabe
  4. Chung-Cheng Chiu
  5. Yu Zhang
  6. Yonghui Wu
  7. Rohit Prabhavalkar
  8. Takaaki Hori

In Machine Translation

  1. Juan Pino
  2. Changhan Wang
  3. Yun Tang
  4. Anne Wu
  5. Xutai Ma
  6. Jiatao Gu
  7. Philipp Koehn
  8. Matteo Negri

Citations in the corpus