Trainable Greedy Decoding for Neural Machine Translation

By Jiatao Gu, Kyunghyun Cho, Victor O. K. Li. 2017

79 citations recorded by the source corpus.

Recent research in neural machine translation has largely focused on two aspects; neural network architectures and end-to-end learning algorithms. The problem of decoding, however, has received relatively little attention from the research community. In this paper, we solely focus on the problem of decoding given a trained neural machine translation model. Instead of trying to build a new decoding algorithm for any specific decoding objective, we propose the idea of trainable decoding algorithm in which we train a decoding algorithm to find a translation that maximizes an arbitrary decoding objective. More specifically, we design an actor that observes and manipulates the hidden state of the neural machine translation decoder and propose to train it using a variant of deterministic policy gradient. We extensively evaluate the proposed algorithm using four language pairs and two decoding objectives and show that we can indeed train a trainable greedy decoder that generates a better translation (in terms of a target decoding objective) with minimal computational overhead.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

GLTR: Statistical Detection and Visualization of Generated Text (2019, methodology): “…of our methods is that to generate natural looking text, most systems sample from the head of the distribution, e.g., through max sampling (Gu et al., 2017), k-max sampling (Fan et al., 2018), beam search (Chorowski and Jaitly, 2016; Shao et al., 2017), temperature-modulated sampling…”
Plug and Play Language Models: A Simple Approach to Controlled Text Generation (2019, background): “In earlier works Gu et al. (2016; 2017); Chen et al. (2018) explored the idea of using a small neural network to steer an LM. Yu et al. (2016), and more recently Yu et al. (2019); Yee et al. (2019); Ng et al. (2019), leveraged the Shannon Noisy Channel Theory (Shannon, 1948) for improving…”
Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting (2018, background): “Training a neural network to use another fixed network has been investigated in machine translation for better decoding (Gu et al., 2017a) and real-time translation (Gu et al., 2017b).”
Neural Machine Translation: A Review (2019, background): “which modifies the hidden states in an already trained model [189, 191].”
A Stable and Effective Learning Strategy for Trainable Greedy Decoding (2018, methodology): “Training To overcome the severe instability re-ported by Gu et al. (2017), we introduce the use of a pseudo-parallel corpus generated from the underlying NMT model (Gao and He, 2013; Auli and Gao, 2014; Kim and Rush, 2016; Chen et al., 2017; Freitag et al., 2017; Zhang et al., 2017) for actor…”
Convolutional Sequence to Sequence Model with Non-Sequential Greedy Decoding for Grapheme to Phoneme Conversion (2018, methodology): “Unfortunately, while there is room for improvement in the decoding methods, there have only been a few studies focusing on them [19, 20].”
Offline RL for Natural Language Generation with Implicit Language Q Learning (2022, background): “A number of prior works have explored combining online RL methods with language models for natural language tasks such as machine translation or summarization [19, 20, 21, 22, 23, 24, 25, 26, 27, 28].”
Greedy Search with Probabilistic N-gram Matching for Neural Machine Translation (2018, background): “Lamb et al. (2016); Gu et al. (2017); Ma et al. (2018) respectively use the discriminator, critic and bag-of-words target as sequence-level training objectives, all of which are directly connected to the generation model and hence enable direct gradient update.”
Steering Language Models With Activation Engineering (2023, unknown): “…overall model performance (Hase et al. 2023; Qi et al. 2023; Brown et al. 2023); • Intervening at decoding , as with guided or trainable decoding (Gu et al. 2017; Grover et al. 2019; see Zhang et al. 2022a for an overview of controlled generation and Jin et al. 2022 for textual style transfer);…”
Comparison of Diverse Decoding Methods from Conditional Language Models (2019, background): “Extensions to NPAD have sought to learn the direction in which to manipulate the hidden states 2 We present the beam search algorithm in the appendix. using an arbitrary decoding objective (Gu et al., 2017).”
Stack-Captioning: Coarse-to-Fine Learning for Image Captioning (2017, background): “Sec-ondly, there is an exposure bias between the training and the testing (Ranzato et al., 2016; Wiseman and Rush, 2016; Gu et al., 2017a).”
Exposure Bias versus Self-Recovery: Are Distortions Really Incremental for Autoregressive Text Generation? (2019, background): “Therefore, it is completely possible that a non-MLE training objective can lead to better generation performance (Lu et al., 2018; Huszár, 2015; Welleck et al., 2020; He and Glass, 2020; Gu et al., 2017).”