Organizing recurrent network dynamics by task-computation to enable continual learning

98 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Natural continual learning: success is a journey, not (just) a destination (2021, methodology): “While the online Laplace approximation has been applied successfully in several continual learning settings (Kirkpatrick et al., 2017; Ritter et al., 2018), it has also been found to perform sub-optimally on a range of problems (van de Ven and Tolias, 2018; Duncker et al., 2020).”
Online Continual Learning in Image Classification: An Empirical Survey (2021, background): “Meanwhile, [137] proposes a learning rule to preserve network dynamics within subspaces for previous tasks and encourage interfering dynamics to explore orthogonal subspaces when learning new tasks.”
Continual learning in recurrent neural networks (2020, background): “In concurrent work, Duncker et al. (2020) introduced a learning rule which aims to optimize the use of the activity-deﬁned subspace in RNNs learning multiple tasks.”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, background): “This can be achieved by exploiting the singular vector space of the activations of the network, which applies to linear and convolutional layers (Saha et al., 2021; Deng et al., 2021), as well as recurrent layers (Duncker et al., 2020).”