Task Loss Estimation for Sequence Prediction

By Dzmitry Bahdanau, Dmitriy Serdyuk, Philemon Brakel, Nan Rosemary Ke, Jan Chorowski, Aaron C. Courville, Yoshua Bengio. 2015

33 citations recorded by the source corpus.

Often, the performance on a supervised machine learning task is evaluated with a emph{task loss} function that cannot be optimized directly. Examples of such loss functions include the classification error, the edit distance and the BLEU score. A common workaround for this problem is to instead optimize a emph{surrogate loss} function, such as for instance cross-entropy or hinge loss. In order for this remedy to be effective, it is important to ensure that minimization of the surrogate loss results in minimization of the task loss, a condition that we call emph{consistency with the task loss}. In this work, we propose another method for deriving differentiable surrogate losses that provably meet this requirement. We focus on the broad class of models that define a score for every input-output pair. Our idea is that this score can be interpreted as an estimate of the task loss, and that the estimation error may be used as a consistent surrogate loss. A distinct feature of such an approach is that it defines the desirable value of the score for every input-output pair. We use this property to design specialized surrogate losses for Encoder-Decoder models often used for sequence prediction tasks. In our experiment, we benchmark on the task of speech recognition. Using a new surrogate loss instead of cross-entropy to train an Encoder-Decoder speech recognizer brings a significant ~13% relative improvement in terms of Character Error Rate (CER) in the case when no extra corpora are used for language modeling.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Imputer: Sequence Modelling via Imputation and Dynamic Programming (2020, background): “Three popular types of generative models have been developed for end-to-end speech recognition applications: CTC (Graves et al., 2006; Graves & Jaitly, 2014), RNN-T (Graves, 2012; Graves et al., 2013) and seq2seq (Chan et al., 2016; Bahdanau et al., 2016a).”
Reward Augmented Maximum Likelihood for Neural Structured Prediction (2016, methodology): “Our experiments demonstrate hat the RAML approach considerably outperforms ML baseline on both speech recognition andmachine translation tasks.”
Very deep convolutional networks for end-to-end speech recognition (2016, methodology): “Recently, very deep CNNs architectures [15] have also been shown to be successful in ASR [16, 17], using more non-linearities, but fewer parameters.”
Latent Sequence Decompositions (2016, methodology): “On the Wall Street Journal speech recognition task, the sequence-to-sequence character model baseline achieves 14.8% WER while the LSD model achieves 12.9%.”
Optimal Completion Distillation for Sequence Learning (2018, methodology): “…adopted in both machine translation (Bahdanau et al., 2015; Wu et al., 2016; Hassan et al., 2018) and speech recognition systems (Chan et al., 2016; Bahdanau et al., 2016a; Chiu et al., 2017) to achieve impressive performance beyond traditional multi-stage pipelines (Koehn et al., 2007; Povey et…”
A Fully Differentiable Beam Search Decoder (2019, methodology): “Also related, several works are targeting the loss/evaluation mismatch (and sometimes exposure bias) through reinforcement learning (policy gradient) (Bahdanau et al., 2016b; Ranzato et al., 2016) even in speech recognition (Zhou et al., 2018).”
Twin Networks: Matching the Future for Sequence Generation (2017, methodology): “For this reason, they usually act as better feature extractors than the unidirectional counterpart and have been successfully used in a myriad of tasks, e.g. in machine translation (Bahdanau et al., 2015), question answering (Chen et al., 2017) and sequence labeling (Ma & Hovy, 2016).”
On Online Attention-Based Speech Recognition and Joint Mandarin Character-Pinyin Training (2016, background): “8% WER trained with Task Loss Estimation (TLE) [23].”
Token-wise Training for Attention Based End-to-end Speech Recognition (2019, unknown): “introduced policy gradient [6] to A-E2E model training [7].”
TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speed (2022, background): “As an impressive example of unsupervised learning, deep generative models have exhibited powerful modeling performance across a wide range of tasks, including variational autoencoders (VAE) [1], Generative Adversarial Nets (GAN) [2], autoregressive models [3] and Flow-based models [4, 5].”
Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model (2016, background): “Much of the recent advances in conditional recurrent language model have focused either on network architectures (e.g., [1]), learning algorithms (e.g., [4, 22, 2]) or novel applications (see [9] and references therein).”
Non-Monotonic Sequential Text Generation (2019, methodology): “Furthermore, even with a fixed generation order, RL has proven to be difficult without partially relying on supervised learning (Ranzato et al., 2015; Bahdanau et al., 2015; 2016).”