Sequence-to-Sequence Learning as Beam-Search Optimization

By Sam Wiseman, Alexander M. Rush. 2016

625 citations recorded by the source corpus.

Sequence-to-Sequence (seq2seq) modeling has rapidly become an important general-purpose NLP tool that has proven effective for many text-generation and sequence-labeling tasks. Seq2seq builds on deep neural language modeling and inherits its remarkable accuracy in estimating local, next-word distributions. In this work, we introduce a model and beam-search training scheme, based on the work of Daume III and Marcu (2005), that extends seq2seq to learn global sequence scores. This structured approach avoids classical biases associated with local training and unifies the training loss with the test-time usage, while preserving the proven model architecture of seq2seq and its efficient training approach. We show that our system outperforms a highly-optimized attention-based seq2seq system and other baselines on three different sequence to sequence tasks: word ordering, parsing, and machine translation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

In Model Architectures

Citations in the corpus

Discriminative Adversarial Search for Abstractive Summarization (2020, neutral): “Several works have focused on alleviating this issue, proposing to optimize a sequence level metric such as BLEU or ROUGE: Wiseman & Rush (2016) used beam search optimisation while Ranzato et al. (2015) framed text generation as a reinforcement learning problem, using the chosen metric as reward.”
An Actor-Critic Algorithm for Sequence Prediction (2016, methodology): “IWSLT 2014 with a bidirectional GRU encoder In order to compare our results with those reported by Wiseman & Rush (2016) we repeated our IWSLT 2014 investigation with a different encoder, a bidirectional RNN with 256 GRU units.”
Deep Neural Solver for Math Word Problems (2017, methodology): “With the framework of seq2seq learning Sutskev-er et al. (2014); Wiseman and Rush (2016), recent advances in neural machine translation (N-MT) Bahdanau et al. (2014); Cho et al. (2014) and neural responding machine (NRM) Shang et al. (2015) have demonstrated the power of recurrent neural networks…”
Towards Better Decoding and Language Model Integration in Sequence to Sequence Models (2016, background): “Global normalization has yielded important benefits on many NLP tasks including parsing and translation [32, 33].”
Reward Augmented Maximum Likelihood for Neural Structured Prediction (2016, methodology): “see large margin, search-based [8, 39], and expected risk optimization methods).”
Maximum-Likelihood Augmented Discrete Generative Adversarial Networks (2017, methodology): “…Bias , especially in sequence-to-sequence learning framework, which obstacles the MLE trained models to be optimized globally (Andor et al., 2016; Wiseman & Rush, 2016) To addresses the abovementioned issues in training auto-regressive models, we propose to formulate the problem under the…”
Classical Structured Prediction Losses for Sequence to Sequence Learning (2017, methodology): “Next, we compare classical sequence-level training to the recently proposed Beam Search Optimization (Wiseman and Rush, 2016).”
Optimal Completion Distillation for Sequence Learning (2018, methodology): “Beam Search Optimization (Wiseman and Rush, 2016) and Edit-based Minimum Bayes Risk (EMBR) (Prabhavalkar et al.”
Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems (2018, background): “In future works, several methods could be applied (e.g. Reinforcement Learning (Ranzato et al., 2016), Beam Search (Wiseman and Rush, 2016)) to improve both responses relevance and entity F1 score.”
What Do Recurrent Neural Network Grammars Learn About Syntax? (2016, methodology): “Our experiment on the importance of composition function was motivated by Vinyals et al. (2015) and Wiseman and Rush (2016), who achieved competitive parsing accuracy without explicit composition.”
Quark: Controllable Text Generation with Reinforced Unlearning (2022, methodology): “Recent works have explored applying RL on tasks such as question-answering [84, 85, 47, 83, 84], summarization [58, 53, 70, 60, 17, 51], and machine translation [58, 87, 79, 82, 81, 13, 66, 5, 49].”
Improving Abstraction in Text Summarization (2018, background): “Bengio et al. (2015) and Wiseman and Rush (2016) propose an alternate method that exposes the network to the test dynamics during training.”
Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement (2019, background): “Although typically a test-time procedure, there are works that include beam search in the training loop (Daum ´ e et al., 2009; Wiseman & Rush, 2016; Edunov et al., 2018b; Negrinho et al., 2018; Edunov et al., 2018a) for training sequence models on the sequence level (Ranzato et al., 2016; Bahdanau…”
Adversarial Generation of Natural Language (2017, background): “Exposure bias occurs when a model is only trained conditioned on ground-truth contexts and is not exposed to its own errors (Wiseman and Rush, 2016).”