Training recurrent networks online without backtracking

By Yann Ollivier, Guillaume Charpiat. 2015

42 citations recorded by the source corpus.

We introduce the "NoBackTrack" algorithm to train the parameters of dynamical systems such as recurrent neural networks. This algorithm works in an online, memoryless setting, thus requiring no backpropagation through time, and is scalable, avoiding the large computational and memory cost of maintaining the full gradient of the current state with respect to the parameters. The algorithm essentially maintains, at each time, a single search direction in parameter space. The evolution of this search direction is partly stochastic and is constructed in such a way to provide, at every time, an unbiased random estimate of the gradient of the loss function with respect to the parameters. Because the gradient estimate is unbiased, on average over time the parameter is updated as it should. The resulting gradient estimate can then be fed to a lightweight Kalman-like filter to yield an improved algorithm. For recurrent neural networks, the resulting algorithms scale linearly with the number of parameters. Preliminary tests on a simple task show that the stochastic approximation of the gradient introduced in the algorithm does not seem to introduce too much noise in the trajectory, compared to maintaining the full gradient, and confirm the good performance and scalability of the Kalman-like version of NoBackTrack.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Decoupled Neural Interfaces using Synthetic Gradients (2016, methodology): “Although we report results only with LSTMs, we have found DNI to work similarly for vanilla RNNs and Leaky RNNs (Ollivier & Charpiat, 2015).”
Reviving and Improving Recurrent Back-Propagation (2018, methodology): “Other efforts have been made to develop alternatives to BPTT. NoBackTrack (Ollivier et al., 2015) maintains an online estimate of the gradient via the random rank-one reduction technique.”
Sparse Attentive Backtracking: Temporal CreditAssignment Through Reminding (2018, methodology): “Completely different approaches are possible but are not currently close to BPTT in terms of learning performance on large networks, such as methods based on the online estimation of gradients (Ollivier et al., 2015).”
Equilibrium Propagation: Bridging the Gap between Energy-Based Models and Backpropagation (2016, methodology): “An encouraging direction is that proposed by Ollivier et al. (2015); Tallec and Ollivier (2017), which shows that computationally efﬁcient estimators of the gradient can be obtained using a forward method (online estimation of the gradient), which avoids the need to store all past states in…”
Sparse Attentive Backtracking: Long-Range Credit Assignment in Recurrent Networks (2017, methodology): “One approach (Ollivier et al., 2015) attempts to solve this problem by estimating gradients using an approximation to forward mode automatic differentiation instead of backpropagation.”
Understanding and correcting pathologies in the training of learned optimizers (2018, methodology): “Existing methods have been proposed to address the bias of TBPTT but come at the cost of increased variance or computational complexity (Williams & Zipser, 1989; Ollivier et al., 2015; Tallec & Ollivier, 2017).”
Unbiasing Truncated Backpropagation Through Time (2017, background): “1 NoBackTrack [OTC15] and Unbiased Online Recurrent Optimization (UORO) [TO17] both scalably provide unbiased online recurrent learning algorithms.”