Learning to Translate in Real-time with Neural Machine Translation

By Graham Neubig, Kyunghyun Cho, Jiatao Gu, Victor O. K. Li. 2016

240 citations recorded by the source corpus.

Translating in real-time, a.k.a.simultaneous translation, outputs translation words before the input sentence ends, which is a challenging problem for conventional machine translation methods. We propose a neural machine translation (NMT) framework for simultaneous translation in which an agent learns to make decisions on when to translate from the interaction with a pre-trained NMT environment. To trade off quality and delay, we extensively explore various targets for delay and design a method for beam-search applicable in the simultaneous MT setting. Experiments against state-of-the-art baselines on two language pairs demonstrate the efficacy of the proposed framework both quantitatively and qualitatively.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Plug and Play Language Models: A Simple Approach to Controlled Text Generation (2019, background): “In earlier works Gu et al. (2016; 2017); Chen et al. (2018) explored the idea of using a small neural network to steer an LM. Yu et al. (2016), and more recently Yu et al. (2019); Yee et al. (2019); Ng et al. (2019), leveraged the Shannon Noisy Channel Theory (Shannon, 1948) for improving…”
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (2018, background): “Training a neural network to use another ﬁxed network has been investigated in machine translation for better decoding (Gu et al., 2017a) and real-time translation (Gu et al., 2017b).”
The Neural Noisy Channel (2016, background): “Another trend of work that is related to our model is the investigation of making online prediction for machine translation (Gu et al., 2016; Grissom II et al., 2014; Sankaran et al., 2010) and speech recognition (Hwang & Sung, 2016; Jaitly et al., 2016).”
Thinking Slow about Latency Evaluation for Simultaneous Machine Translation (2019, background): “…et al. (2018) report that AP is not sufﬁciently sensitive to detect their improvements to a reinforcement learning approach to simultaneous MT. Gu et al. (2017) use AP, and also introduce the position-wise latency metric Consecutive Wait (CW), which measures the number of consecutive reads…”
STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework (2018, methodology): “1 Existing Metrics: CW and AP Consecutive Wait (CW) (Gu et al., 2017) is the number of source words waited between two target words.”
Survey on reinforcement learning for language processing (2021, background): “[37] propose an NMT model for real-time translation, where a task-specific neural network learns to decide which actions to take (i.”
Learning to Repeat: Fine Grained Action Repetition for Deep Reinforcement Learning (2017, unknown): “For instance, Gu et al. (2016) and Satija & Pineau (2016) explore Real Time Neural Machine Translation where the action at every time step is to decide whether to output a new token in the target language or not based on current context.”
Trainable Greedy Decoding for Neural Machine Translation (2017, background): “In simultaneous translation/interpretation, which has recently been studied in the context of neural machine translation (Gu et al., 2016), the decoding objective is formulated as a trade-off between the translation quality and delay.”
Incremental Text to Speech for Neural Sequence-to-Sequence Models using Reinforcement Learning (2020, methodology): “The development of these streaming, end-to-end architectures has seen considerable attention for the tasks of automatic speech recognition [7, 8, 9, 10] and machine translation (MT) [11, 12, 13, 14].”
Neural iTTS: Toward Synthesizing Speech in Real-time with End-to-end Neural Text-to-Speech Framework (2019, background): “These studies are widely conducted from parametric models to neural network architecture [4, 5, 6, 7, 8, 9, 10, 11].”
Monotonic Multihead Attention (2019, background): “Gu et al. (2017) introduce an agent which learns to make decisions on when to translate from the interaction with a pre-trained neural machine translation model.”
ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic (2021, background): “Several studies have explored the idea of steering an LM using small neural networks [10, 25].”
SIMULEVAL: An Evaluation Toolkit for Simultaneous Translation (2020, background): “…2002; Post, 2018), a wide variety of latency measurements have been introduced, such as Average Proportion (AP) (Cho and Esipova, 2016), Continues Wait Length (CW) (Gu et al., 2017), Average Lagging (AL) (Ma et al., 2019), Differentiable Average Lag-ging (DAL) (Cherry and Foster, 2019), and so on.”
STACL: Simultaneous Translation with Integrated Anticipation and Controllable Latency (2018, methodology): “4, we compare the BLEU score and AP with the model from (Gu et al., 2017) on test set for English-to-German task.”
Direct simultaneous speech to speech translation (2021, background): “While a fair amount of research has been conducted on simultaneous translation, including text-to-text (Cho & Esipova, 2016; Gu et al., 2017; Ma et al., 2019a; Arivazhagan et al., 2019; Ma et al., 2019c), and speech-to-text (Ren et al., 2020; Ma et al., 2020c; 2021), only a few works have explored…”
What the Future Brings: Investigating the Impact of Lookahead for Incremental Neural TTS (2020, unknown): “Several strategies have been proposed to reduce the latency of a sequence-to-sequence model with input text for neural machine translation [9, 10, 11] or incremental speech translation [12, 13, 14].”