Mitigation of catastrophic forgetting in recurrent neural networks using a Fixed Expansion Layer

20 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

RATT: Recurrent Attention to Transient Tasks for Continual Image Captioning (2020, methodology): “A fixed expansion layer technique was proposed to mitigate forgetting in RNNs in [6].”
Class-Incremental Learning: Survey and Performance Evaluation on Image Classification (2020, methodology): “To address catastrophic forgetting an expansion layer technique for RNNs was proposed [96].”
Continual learning in recurrent neural networks (2020, methodology): “Fixed expansion layers (Coop and Arel, 2012) are another method to limit the plasticity of weights and prevent forgetting, and in RNNs take the form of a sparsely activated layer between consecutive hidden states (Coop and Arel, 2013).”
Continual Learning with Gated Incremental Memories for sequential data processing (2020, methodology): “More recently, RNNs have been exploited in combination with other techniques, such as Fixed Expansion Layer [19], external growing memories [20], Reservoir Computing [21] and backpropagation-free learning [22].”
Continual Learning in Recurrent Neural Networks with Hypernetworks (2020, methodology): “Fixed expansion layers [23] are another method to limit the plasticity of weights and prevent forgetting, and in RNNs take the form of a sparsely activated layer between consecutive hidden states [24].”