Decoupled Neural Interfaces using Synthetic Gradients

By Max Jaderberg, Wojciech M. Czarnecki, Simon Osindero, Oriol Vinyals, Alex Graves, David Silver, Koray Kavukcuoglu. 2016

420 citations recorded by the source corpus.

Training directed neural networks typically requires forward-propagating data through a computation graph, followed by backpropagating error signal, to produce weight updates. All layers, or more generally, modules, of the network are therefore locked, in the sense that they must wait for the remainder of the network to execute forwards and propagate error backwards before they can be updated. In this work we break this constraint by decoupling modules by introducing a model of the future computation of the network graph. These models predict what the result of the modelled subgraph will produce using only local information. In particular we focus on modelling error gradients: by using the modelled synthetic gradient in place of true backpropagated error gradients we decouple subgraphs, and can update them independently and asynchronously i.e. we realise decoupled neural interfaces. We show results for feed-forward models, where every layer is trained asynchronously, recurrent neural networks (RNNs) where predicting one's future gradient extends the time over which the RNN can effectively model, and also a hierarchical RNN system with ticking at different timescales. Finally, we demonstrate that in addition to predicting gradients, the same framework can be used to predict inputs, resulting in models which are decoupled in both the forward and backwards pass -- amounting to independent networks which co-learn such that they can be composed into a single functioning corporation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

In Model Architectures

Citations in the corpus

Emergent Abilities of Large Language Models (2022, neutral): “…inputs (Graves, 2016; Dehghani et al., 2018), using more localized learning strategies than backpropagation through all weights in a neural network (Jaderberg et al., 2017), and augmenting models with external memory (Guu et al., 2020; Borgeaud et al., 2021; Wu et al., 2022b, inter alia ).”
HyperNetworks (2016, background): “The concept of a network interacting with another network is central to the work of (Jaderberg et al., 2016; Andrychowicz et al., 2016), and especially (Denil et al., 2013; Yang et al., 2015; Bertinetto et al., 2016; De Brabandere et al., 2016), where certain parameters in a convolutional network…”
The Reversible Residual Network: Backpropagation Without Storing Activations (2017, methodology): “The decoupled neural interface [15] updates each weight matrix using a gradient approximation, termed the synthetic gradient, computed based on only the node’s activations instead of the global network error.”
Advances and Open Problems in Federated Learning (2019, background): “The ideas in [213, 207], where the authors break the dependencies between partial networks and reduced total centralized training time by parallelizing the computations in different parts, can be relevant here as well.”
Training Neural Networks Using Features Replay (2018, methodology): “The synthesizer network in [13] is so small that it cannot learn an accurate approximation of the error gradient when the network is deep.”
Few-Shot and Continual Learning with Attentive Independent Mechanisms (2021, methodology): “SIB is composed of two works: synthetic gradient modeling [21] and a feature averaging classifier [15].”
The Mechanics of n-Player Differentiable Games (2018, background): “Copyright 2018 by the author(s). optimization (Pfau & Vinyals, 2016), synthetic gradients (Jaderberg et al., 2017), hierarchical reinforcement learning (Wayne & Abbott, 2014; Vezhnevets et al., 2017), curiosity (Pathak et al., 2017), and imaginative agents (Racani`ere et al., 2017).”
A solution to the learning dilemma for recurrent networks of spiking neurons (2019, background): “For example, the combination of eligibility traces from e-prop with synthetic gradients from Jaderberg et al. (2016) substantially improves performance of LSTM networks for difficult machine learning problems such as the copy-repeat task and the Penn Treebank word prediction task (Bellec et al.,…”
Greedy Layerwise Learning Can Scale to ImageNet (2018, background): “Finally, end-to-end back-propagation can be inefficient (Jaderberg et al., 2016; Salimans et al., 2017) in terms of computation and memory resources.”
Empirical Bayes Transductive Meta-Learning with Synthetic Gradients (2020, methodology): “In this paper, we propose to use synthetic gradient (Jaderberg et al., 2017) to enable transduction, such that the variational posterior is implemented as a function of the labeled set dlt and the unlabeled set xt.”
Gradient Adversarial Training of Neural Networks (2018, methodology): “We can also replace direct gradient tensor evaluation with synthetic gradients [14] for efﬁciency.”
State Aware Imitation Learning (2017, background): “Finally, we would like to point out the similarities our work shares with meta learning techniques that learn the gradients (e.g. [12]) or determine the weight updates (e.g. [22], [8]) for a neural network.”
Learning to Predict Gradients for Semi-Supervised Continual Learning (2022, background): “Jaderberg et al . propose a learning framework with the synthetic gradients to allow layers to be updated in an asynchronous fashion [45].”
Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout (2020, methodology): “Apart from the traditional optimization methods [8, 19, 32, 33, 49], there has been a research thrust on developing different ways to apply gradients to deep networks [2, 7, 10, 15, 36, 45, 50].”
Associated Learning: Decomposing End-to-End Backpropagation Based on Autoencoders and Target Propagation (2019, background): “, the weights must be updated layer by layer), training a deep neural network can be extremely inefficient (Jaderberg et al., 2016).”