Professor Forcing: A New Algorithm for Training Recurrent Networks

By Anirudh Goyal, Alex Lamb, Ying Zhang, Saizheng Zhang, Aaron C. Courville, Yoshua Bengio. 2016

692 citations recorded by the source corpus.

The Teacher Forcing algorithm trains recurrent networks by supplying observed sequence values as inputs during training and using the network’s own one-step-ahead predictions to do multi-step sampling. We introduce the Professor Forcing algorithm, which uses adversarial domain adaptation to encourage the dynamics of the recurrent network to be the same when training the network and when sampling from the network over multiple time steps. We apply Professor Forcing to language modeling, vocal synthesis on raw waveforms, handwriting generation, and image generation. Empirically we find that Professor Forcing acts as a regularizer, improving test likelihood on character level Penn Treebank and sequential MNIST. We also find that the model qualitatively improves samples, especially when sampling for a large number of time steps. This is supported by human evaluation of sample quality. Trade-offs between Professor Forcing and Scheduled Sampling are discussed. We produce T-SNEs showing that Professor Forcing successfully makes the dynamics of the network during training and sampling more similar.

Subfields: Artificial Intelligence, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Optimization & Training

Citations in the corpus

MaskGAN: Better Text Generation via Filling in the ______ (2018, background): “Professor Forcing (Lamb et al., 2016) is an alternative to training an RNN with teacher forcing by using a discriminator to discriminate the hidden states of a generator RNN that is conditioned on real and synthetic samples.”
Discriminative Adversarial Search for Abstractive Summarization (2020, neutral): “Rather than exclusively targeting exposure bias (as in Scheduled Sampling or Professor Forcing), or relying on automatic metrics as in reinforcement learning approaches, we propose optimizing toward a discriminator instead of discrete metrics: we ﬁnd that besides reducing the exposure bias issue, th”
Teaching Autoregressive Language Models Complex Tasks By Demonstration (2021, methodology): “We refer to this approach as environment forcing by analogy to related concepts such as professor forcing[33] and attention forcing[34], as it allows the state of the environment to impose constraints on the generated text, with the goal of mitigating exposure bias.”
Self-Critical Sequence Training for Image Captioning (2016, background): “Another line of work proposes “Professor-Forcing” [9], a technique that uses adversarial training to encourage the dynamics of the recurrent network to be the same when training conditioned on ground truth previous words and when sampling freely from the network.”
Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments (2017, methodology): “Based on this ground-truth definition, our initial approach to training (‘teacher-forcing’ [26]) uses cross entropy loss at each step to maximize the likelihood of the next ground-truth action given the previous ground-truth action.”
Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization (2018, background): “Adversarial learning [6, 26] has been shown to be successful in dialog generation, translation, image captioning and a series of natural language generation tasks [7, 12, 27, 28, 29, 30, 31, 32, 33, 34].”
Learning Latent Dynamics for Planning from Pixels (2018, methodology): “Therefore, we take inspiration from Amos et al. (2018) and earlier related ideas (Chiappa et al., 2017; Villegas et al., 2017; Lamb et al., 2016), and train all multi-step predictions of the model.”
Unsupervised Machine Translation Using Monolingual Corpora Only (2017, methodology): “This algorithm, called Professor forcing, was initially introduced by Lamb et al. (2016) to encourage the dynamics of the decoder observed during inference to be similar to the ones observed at training time.”
Adversarial Learning for Neural Dialogue Generation (2017, methodology): “The adversarial REINFORCE algorithm consists of two components: a generative model G and a discriminative model D .”
Towards Principled Methods for Training Generative Adversarial Networks (2017, background): “(Denton et al., 2015; Radford et al., 2015; Salimans et al., 2016; Lamb et al., 2016; Wu et al., 2016) However, they still remain remarkably difﬁcult to train, with most current papers dedicated to heuristically ﬁnding stable architectures.”
Style Transfer from Non-Parallel Text by Cross-Alignment (2017, methodology): “In our work, we employ the Professor-Forcing algorithm (Lamb et al., 2016) which was originally proposed to close the gap between teacher-forcing during training and self-feeding during testing for recurrent networks.”
Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning (2017, background): “The Deep Learning community has recently proposed Generative Adversarial Networks (GANs) [29, 68, 70], which are still being intensively developed [3, 9, 31, 41, 54].”
Adversarial Text Generation Without Reinforcement Learning (2018, methodology): “To reduce the instability of sequence prediction with recurrent neural networks, Lamb et al. (2016) use an adversarial network to encourage similar system behavior during training and prediction.”
FRAGE: Frequency-Agnostic Word Representation (2018, background): “[21] develops a discriminator to classify whether a sentence is created by human or generated by a model.”
A Multi-Horizon Quantile Recurrent Forecaster (2017, methodology): “Bengio et al, 2015 and Lamb et al, 2016 pointed out that a carefully designed training scheme is needed when the Recursive strategy is applied with RNN, to avoid the discrepancy be-tween consuming actual data versus estimates during prediction, since the latter leads to error accumulation.”
A Survey on Neural Speech Synthesis (2021, background): “[96] leverage professor forcing [89] to alleviate the mismatch between the different distributions of real and predicted data.”
Maximum-Likelihood Augmented Discrete Generative Adversarial Networks (2017, methodology): “Lamb et al. (2016) exploits adversarial domain adaption to regularize the training of recurrent neural networks.”
Boundary-Seeking Generative Adversarial Networks (2017, background): “There have also been some improvements recently on training GANs on language data by rephrasing the problem into a GAN over some continuous space (Lamb et al., 2016; Kim et al., 2017; Gulrajani et al., 2017).”
Adversarial Feature Matching for Text Generation (2017, background): “Recent work (Lamb et al., 2016) has incorporated an additional discriminator to train a sequence-to-sequence language model that better preserves ar X iv :1 70 6.”