Catastrophic interference in neural networks

By Stephan Lewandowsky, Shu-Chen Li. 1995

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Continual Lifelong Learning with Neural Networks: A Review (2018, background): “Lopez-Paz and Ranzato (2017) proposed the Gradient Episodic Memory (GEM)model that yields positive transfer of knowledge to previous tasks.”
Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills (2015, methodology): “Orthogonalization techniques mitigate interference between tasks by reducing their representational overlap in input neurons (via manually designed preprocessing) and by encouraging sparse hidden-neuron activations [30–32].”
Neural networks with a self-refreshing memory: Knowledge transfer in sequential learning tasks without catastrophic forgetting (2000, background): “…in sequential learning tasks (Hetherington and Seidenberg Connection Science, Vol. 12, No. 1, 2000, 1–19 1989, McCloskey and Cohen 1989, Kortge 1990, Ratcliff 1990, Lewandowsky 1991, 1994, Murre 1992, French 1992, 1994, 1997, McRae and Hetherington 1993, Lewandowsky and Li 1995, McClelland et al .”
Self-refreshing memory in artificial neural networks: learning temporal sequences without catastrophic forgetting (2004, background): “…McCloskey and Cohen 1989, Kortge 1990, Ratcliff 1990, Lewandowsky 1991, 1994, Murre 1992, French 1992, 1994, 1997, McRae and Hetherington 1993, Lewandowsky and Li 1995, McClelland et al. 1995, Robins 1995, 1996, Sharkey and Sharkey 1995, Ans and Rousset 1997, 2000, Robins and McCallum 1998,…”
The consolidation of learning during sleep: comparing the pseudorehearsal and unlearning accounts (1999, unknown): “…issues—these include: McCloskey and Cohen (1989), Hetherington and Seidenberg (1989), Ratcliff (1990), Lewandowsky (1991), Murre (1992a), French (1992) (1994) (1997), McRae and Hetherington (1993), Lewandowsky and Li (1995), Sharkey and Sharkey (1995), Robins (1995) (1996), Frean and Robins (1998).”
Effects of Auxiliary Knowledge on Continual Learning (2022, background): “[18] first pioneered the idea to reduce forgetting by orthogonalizing feature representations.”
Combatting climate change misinformation: Evidence for longevity of inoculation and consensus messaging effects (2020, unknown): “Researchers have found that inoculation interventions strengthen associative memory networks, but this network could be subject to interference (Hardt et al., 2013; Lewandowsky & Li, 1995; Pfau et al., 1997, 2005).”
Entropy-based Sample Selection for Online Continual Learning (2021, background): “In such a setting, DNNs typically suffer from a phenomenon called catastrophic forgetting [4], [5], a rapid decrease in performance on all but the most recent task.”