Unbiasing Truncated Backpropagation Through Time

By Corentin Tallec, Yann Ollivier. 2017

88 citations recorded by the source corpus.

\emph{Truncated Backpropagation Through Time} (truncated BPTT, \cite{jaeger2002tutorial}) is a widespread method for learning recurrent computational graphs. Truncated BPTT keeps the computational benefits of \emph{Backpropagation Through Time} (BPTT \cite{werbos:bptt}) while relieving the need for a complete backtrack through the whole data sequence at every step. However, truncation favors short-term dependencies: the gradient estimate of truncated BPTT is biased, so that it does not benefit from the convergence guarantees from stochastic gradient theory. We introduce \emph{Anticipated Reweighted Truncated Backpropagation} (ARTBP), an algorithm that keeps the computational benefits of truncated BPTT, while providing unbiasedness. ARTBP works by using variable truncation lengths together with carefully chosen compensation factors in the backpropagation equation. We check the viability of ARTBP on two tasks. First, a simple synthetic task where careful balancing of temporal dependencies at different scales is needed: truncated BPTT displays unreliable performance, and in worst case scenarios, divergence, while ARTBP converges reliably. Second, on Penn Treebank character-level language modelling \cite{ptb_proc}, ARTBP slightly outperforms truncated BPTT.

Subfields: Artificial Intelligence, Optimization & Training

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Optimization & Training

Citations in the corpus

Character-Level Language Modeling with Deeper Self-Attention (2018, methodology): “A range of methods have arisen for unbiasing and improving TBTT [41, 20].”
Regularizing and Optimizing LSTM Language Models (2017, methodology): “This linear scaling rule has been noted as important for training large scale minibatch SGD without loss of accuracy (Goyal et al., 2017) and is a component of unbiased truncated backpropagationthrough time (Tallec & Ollivier, 2017).”
Residual Flows for Invertible Generative Modeling (2019, methodology): “This paradigm of estimation has been repeatedly rediscovered and applied in many fields, including solving of stochastic differential equations (McLeish, 2011; Rhee and Glynn, 2012, 2015), ray tracing for rendering paths of light (Arvo and Kirk, 1990), and estimating limiting behavior of optimizatio”
Reviving and Improving Recurrent Back-Propagation (2018, background): “Cora Baseline 39.96 ± 3.4 BPTT 24.48 ± 6.6 TBPTT 46.55 ± 6.4 Uniform-ARTBP 27.88 ± 3.2 TPoisson-ARTBP 42.22 ± 7.1 RBP 29.25 ± 3.3 CG-RBP 39.26 ± 6.5 Neumann-RBP 46.63 ± 8.3 Table A3.”
Sparse Attentive Backtracking: Long-Range Credit Assignment in Recurrent Networks (2017, methodology): “The Unbiased Online Recurrent Optimization (UORO) (Tallec & Ollivier, 2017) method gets around this by updating a rank-1 approximation to this gradient tensor, which is shown to keep the estimate of the gradient unbiased, but potentially at the risk of increasing the variance of the gradient estimat”
Brax - A Differentiable Physics Engine for Large Scale Rigid Body Simulation (2021, background): “This sort of truncation is known to bias the gradients, but with the advantage that it can make optimization easier[44].”
Gradients are Not All You Need (2021, methodology): “Another common technique used to control these issues is called truncated backpropogation through time [Werbos, 1990, Tallec and Ollivier, 2017].”
Flexible Diffusion Modeling of Long Videos (2022, background): “And while deep generative models based on recurrent neural networks (RNN) theoretically impose no such conditional independence assumptions, in practice they must be trained over short sequences [11, 25] or with truncated gradients [30].”
Learning Discrete Structures for Graph Neural Networks (2019, methodology): “The bias stems from both the straight-trough estimator and from the truncation procedure introduced in lines 11-13 (Tallec & Ollivier, 2017).”
Understanding and correcting pathologies in the training of learned optimizers (2018, methodology): “This not only yields computational savings, at the cost of increased bias (Tallec & Ollivier, 2017), but also limits exploding gradients which emerge from too many iterated non-linear function applications (Pascanu et al., 2013; Parmas et al., 2018).”