PathNet: Evolution Channels Gradient Descent in Super Neural Networks

By Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David R Ha, Andrei A. Rusu, Alexander Pritzel, Daan Wierstra. 2017

965 citations recorded by the source corpus.

For artificial general intelligence (AGI) it would be efficient if multiple users trained the same giant neural network, permitting parameter reuse, without catastrophic forgetting. PathNet is a first step in this direction. It is a neural network algorithm that uses agents embedded in the neural network whose task is to discover which parts of the network to re-use for new tasks. Agents are pathways (views) through the network which determine the subset of parameters that are used and updated by the forwards and backwards passes of the backpropogation algorithm. During learning, a tournament selection genetic algorithm is used to select pathways through the neural network for replication and mutation. Pathway fitness is the performance of that pathway measured according to a cost function. We demonstrate successful transfer learning; fixing the parameters along a path learned on task A and re-evolving a new population of paths for task B, allows task B to be learned faster than it could be learned from scratch or after fine-tuning. Paths evolved on task B re-use parts of the optimal path evolved on task A. Positive transfer was demonstrated for binary MNIST, CIFAR, and SVHN supervised learning classification tasks, and a set of Atari and Labyrinth reinforcement learning tasks, suggesting PathNets have general applicability for neural network training. Finally, PathNet also significantly improves the robustness to hyperparameter choices of a parallel asynchronous reinforcement learning algorithm (A3C).

Subfields: Artificial Intelligence, Electrical and Electronic Engineering, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Electrical and Electronic Engineering

In Model Architectures

Citations in the corpus

The Natural Language Decathlon: Multitask Learning as Question Answering (2018, methodology): “, 2017), rehearsing on old tasks (Robins, 1995), using adaptive memory buffers (Gepperth and Karaoguz, 2016), finding task-specific paths through networks (Fernando et al., 2017), and packing new tasks into already trained networks (Mallya and Lazebnik, 2017).”
Recurrent World Models Facilitate Policy Evolution (2018, background): “Recent works [14, 26, 73, 94] have demonstrated that ES is a viable alternative to traditional Deep RL methods on many strong baselines.”
Regularized Evolution for Image Classifier Architecture Search (2018, background): “Some even forewent the idea of individual architectures (Saxena & Verbeek, 2016; Fernando et al., 2017) and some used evolution to train a single architecture (Jaderberg et al.”
Continual Lifelong Learning with Neural Networks: A Review (2018, background): “PathNet’s authors showed that incrementally learning new tasks sped up the training of subsequently learned supervised and reinforcement learning tasks; however, they did not measure performance on the original task to determine if catastrophic forgetting occurred.”
A Comprehensive Survey of Continual Learning: Theory, Method and Application (2023, methodology): “PathNet [115] and RPSNet [348] pre-allocate multiple parallel”
Knowledge Flow: Improve Upon Your Teachers (2019, methodology): “In particular, we follow Rusu et al. (2016b); Fernando et al. (2017) and compare on the same Atari games.”
DisCoRL: Continual Reinforcement Learning via Policy Distillation (2019, background): “The third category of strategies, dy-2 namic network architectures , maintains past knowledge thanks to architectural modiﬁcations while learning [Rusu et al., 2016a, Fernando et al., 2017, Li and Hoiem, 2018, Fernando et al., 2017].”
MaskConnect: Connectivity Learning by Gradient Descent (2018, neutral): “Evolutionary search has been proposed as an interesting framework to learn both the structure as well as the connections in a neural network [16,17,18,19,20,21,22,23,24].”
Connectivity Learning in Multi-Branch Networks (2017, methodology): “owever, these prior methods operate in stages where initially the network with full connectivity is learned and then connections are greedily removed according to an importance criterion. In PathNet (Fernando et al., 2017), the connectivity within a given architecture was searched via evolution. Com”
Recurrent Independent Mechanisms (2019, background): “Modularity and Neural Networks: A neural network is composed of several neural modules, where each module is meant to perform a distinct function, and hence can be seen as a combination of experts (Jacobs et al., 1991; Bottou & Gallinari, 1991; Ronco et al., 1997; Reed & De Freitas, 2015; Andreas et”
Overcoming catastrophic forgetting with hard attention to the task (2018, methodology): “However, unlike PathNet, the paths in HAT are not based on modules, but on single units.”
FearNet: Brain-Inspired Model for Incremental Learning (2017, methodology): “PathNet uses an evolutionary algorithm to ﬁnd the optimal path through a large DNN, and then freezes the weights along that path (Fernando et al., 2017).”
Deep Generative Dual Memory Network for Continual Learning (2017, methodology): “, 2016) and Pathnets (Fernando et al., 2017) directly freeze important pathways in neural networks, which eliminates forgetting altogether but requires growing the network after each task and can cause the architecture complexity to grow with the number of tasks.”
Measuring Catastrophic Forgetting in Neural Networks (2017, methodology): “Google’s PathNet model uses a genetic algorithm to find the optimal path through a large DCNN and then locks that path to preserve it [2].”
Gradient Episodic Memory for Continual Learning (2017, methodology): “This relates to methods that leverage a modular structure of the network with primitives that can be shared across tasks [Rusu et al., 2016, Fernando et al., 2017, Aljundi et al., 2016, Denoyer and Gallinari, 2015, Eigen et al., 2014].”
CORe50: a New Dataset and Benchmark for Continuous Object Recognition (2017, background): “…Forgetting [13], Progressive Neural Networks [23], Active Long Term Memory Networks [7], Adaptive Convolutional Neural Network [29], PathNet [5], Incremental Regularized Least Squares [3], Elastic Weight Consolidation [11], Encoder-based Lifelong Learning [28], etc. Elastic Weight…”
Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks (2017, methodology): “A different approach was presented by Fernando et al. (2017), in which evolution determines a subset of pathways through a network that are trained through backpropagation, allowing the same network to learn a variety of different tasks.”
Overcoming Catastrophic Forgetting by Incremental Moment Matching (2017, background): “Recently, the concept of applying a regularization function to a network trained by the old task to learning a new task has received much attention.”