Differentiable Scheduled Sampling for Credit Assignment

By Kartik Goyal, Chris Dyer, Taylor Berg-Kirkpatrick. 2017

40 citations recorded by the source corpus.

We demonstrate that a continuous relaxation of the argmax operation can be used to create a differentiable approximation to greedy decoding in sequence-to-sequence (seq2seq) models. By incorporating this approximation into the scheduled sampling training procedure–a well-known technique for correcting exposure bias–we introduce a new training objective that is continuous and differentiable everywhere and can provide informative gradients near points where previous decoding decisions change their value. By using a related approximation, we also demonstrate a similar approach to sampled-based training. We show that our approach outperforms both standard cross-entropy training and scheduled sampling procedures in two sequence prediction tasks: named entity recognition and machine translation.

Subfields: Artificial Intelligence, Political Science and International Relations, Reinforcement Learning, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Political Science and International Relations

In Reinforcement Learning

In Optimization & Training

Citations in the corpus

Is Reinforcement Learning (Not) for Natural Language Processing?: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization (2022, background): “…Schedule Sampling (SS) (Bengio et al., 2015), Parallel SS (Duckworth et al., 2019), SS for Transformers (Mihaylova & Martins, 2019), Diffential SS (Goyal et al., 2017), LOLS (Lampouras & Vlachos, 2016; Chang et al., 2015), TextGAIL (Wu et al., 2021b), and SEARNN (Leblond et al., 2017), have been…”
Survey of Hallucination in Natural Language Generation (2022, background): “[47] create a diferentiable approximation to greedy decoding that shows a good performance in the NMT task.”
Style Transfer from Non-Parallel Text by Cross-Alignment (2017, background): “, 2016), the training procedure can be effectively optimized through back-propagation (Kusner and Hernández-Lobato, 2016; Goyal et al., 2017).”
Scheduled Sampling for Transformers (2019, methodology): “• Inspired by the work of Goyal et al. (2017), we experiment with passing a mix of the embeddings with softmax with temperature.”
Towards Decoding as Continuous Optimisation in Neural Machine Translation (2017, methodology): “Recently, relaxation techniques have been applied to deep models for training and inference in text classiﬁcation (Belanger and McCallum, 2016; Belanger et al., 2017), and fully differentiable training of sequence-to-sequence models with scheduled-sampling (Goyal et al., 2017).”
Autoregressive Knowledge Distillation through Imitation Learning (2020, unknown): “, 2015) and its extensions (Goyal et al., 2017; Zhang et al., 2019) alleviate exposure bias by replacing some words in the true context with the model’s prediction.”
SEQˆ3: Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive Sentence Compression (2019, methodology): “ST GS reportedly outperforms scheduled sampling (Goyal et al., 2017) and converges faster than reinforcement learning (Havrylov and Titov, 2017).”
Differentiable Sampling with Flexible Reference Word Order for Neural Machine Translation (2019, methodology): “Data We evaluate our approach on IWSLT 2014 German-English (de-en) as prior work (Goyal et al., 2017), as well as two additional tasks: IWSLT 2014 English-German (en-de) and IWSLT”