SEARNN: Training RNNs with Global-Local Losses

By Rémi Leblond, Jean-Baptiste Alayrac, Anton Osokin, Simon Lacoste-Julien. 2017

51 citations recorded by the source corpus.

We propose SEARNN, a novel training algorithm for recurrent neural networks (RNNs) inspired by the "learning to search" (L2S) approach to structured prediction. RNNs have been widely successful in structured prediction applications such as machine translation or parsing, and are commonly trained using maximum likelihood estimation (MLE). Unfortunately, this training loss is not always an appropriate surrogate for the test error: by only maximizing the ground truth probability, it fails to exploit the wealth of information offered by structured losses. Further, it introduces discrepancies between training and predicting (such as exposure bias) that may hurt test performance. Instead, SEARNN leverages test-alike search space exploration to introduce global-local losses that are closer to the test error. We first demonstrate improved performance over MLE on two different tasks: OCR and spelling correction. Then, we propose a subsampling strategy to enable SEARNN to scale to large vocabulary sizes. This allows us to validate the benefits of our approach on a machine translation task.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis (2021, background): “Their samples can still exert artifacts resulting from a sequential modeling of components, and, since these models are always trained by “teacher-forcing”, they are susceptible to an exposure bias [3, 54, 23, 60, 40].”
Is Reinforcement Learning (Not) for Natural Language Processing?: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization (2022, methodology): “Imitation learning for NLP. Algorithms such as Schedule Sampling (SS) (Bengio et al., 2015), Parallel SS (Duckworth et al., 2019), SS for Transformers (Mihaylova & Martins, 2019), Diffential SS (Goyal et al., 2017), LOLS (Lampouras & Vlachos, 2016; Chang et al., 2015), TextGAIL (Wu et al., 2021b), a”
Structured Convolutional Kernel Networks for Airline Crew Scheduling (2021, methodology): “Although this method has achieved impressive results in machine translation (Leblond et al., 2017), its success ultimately depends on the neural network’s ability to model the conditional distribution and it is often sensitive to the order in which input data is processed, particularly in…”
Optimal Completion Distillation for Sequence Learning (2018, methodology): “On the other hand, despite many attempts to mitigate the limitations of Maximum Likelihood Estimation (MLE) (Ranzato et al., 2016; Wiseman and Rush, 2016; Norouzi et al., 2016; Bahdanau et al., 2017; Leblond et al., 2018), MLE is still considered the dominant approach for training seq2seq models.”
Text Generation by Learning from Demonstrations (2020, unknown): “, 2011) and the learning-to-search approach has been applied to RNNs to incorporate losses of sequences deviating from the references (Leblond et al., 2018).”
Generalization in Generation: A closer look at Exposure Bias (2019, background): “, 2016) and learning-to-search (Leblond et al., 2018) have been proposed.”
Deep Structured Prediction with Nonlinear Output Transformations (2018, background): “It has achieved impressive results in machine translation [51, 28], computer vision [38], and multi-label classification [36].”
Differentiable Sampling with Flexible Reference Word Order for Neural Machine Translation (2019, methodology): “Leblond et al. (2018) proposed to complete a predicted preﬁx with all possible reference sufﬁxes and picking the reference sufﬁx that yields the highest BLEU-1 score.”
Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation (2022, background): “, 2020), learning to search (Leblond et al., 2018), energy-based models (Deng et al.”
Parametric Adversarial Divergences are Good Task Losses for Generative Modeling (2017, methodology): “See Leblond et al. (2017) and references therein for more principled approaches to sequence prediction with autoregressive models.”
Seq2Slate: Re-ranking and Slate Optimization with RNNs (2018, methodology): “In many seq2seq applications, using greedy decoding at test time performs better than sampling from the model [e.g., Ranzato et al., 2016, Leblond et al., 2018].”
Non-Monotonic Sequential Text Generation (2019, background): “Despite this, experimentally, it has often been found that simply using the oracle’s state distribution is optimal (Ranzato et al., 2015; Leblond et al., 2018).”
Loss Functions for Multiset Prediction (2017, methodology): “[10] proposed SeaRNN, adapting L2S to modern recurrent models.”
Autoregressive Knowledge Distillation through Imitation Learning (2020, methodology): “Simlarly, SEARNN (Leblond et al., 2017) trains RNNs to iteratively generate sequences with beam search to compute the local loss of a single action during the decoding process.”
EDITOR: An Edit-Based Transformer with Repositioning for Neural Machine Translation with Soft Lexical Constraints (2020, unknown): “As prior work suggests that cost functions close to the cross-entropy loss are better suited to deep neural models than the squared error (Leblond et al., 2018; Cheng et al., 2018), we deﬁne the cost function as the KL divergence be-tween the action distributions given by the model policy and by…”