Generalization Through the Recurrent Interaction of Episodic Memories

By Dharshan Kumaran, James L. McClelland. 2012

351 citations recorded by the source corpus.

In this article, we present a perspective on the role of the hippocampal system in generalization, instantiated in a computational model called REMERGE (recurrency and episodic memory results in generalization). We expose a fundamental, but neglected, tension between prevailing computational theories that emphasize the function of the hippocampus in pattern separation (Marr, 1971; McClelland, McNaughton, & O'Reilly, 1995), and empirical support for its role in generalization and flexible relational memory (Cohen & Eichenbaum, 1993; Eichenbaum, 1999). Our account provides a means by which to resolve this conflict, by demonstrating that the basic representational scheme envisioned by complementary learning systems theory (McClelland et al., 1995), which relies upon orthogonalized codes in the hippocampus, is compatible with efficient generalization—as long as there is recurrence rather than unidirectional flow within the hippocampal circuit or, more widely, between the hippocampus and neocortex. We propose that recurrent similarity computation, a process that facilitates the discovery of higher-order relationships between a set of related experiences, expands the scope of classical exemplar-based models of memory (e.g., Nosofsky, 1984) and allows the hippocampus to support generalization through interactions that unfold within a dynamically created memory space.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Developmental and Educational Psychology, Meta & Continual Learning, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Developmental and Educational Psychology

In Meta & Continual Learning

In Statistical Learning Theory

Citations in the corpus

Continual Lifelong Learning with Neural Networks: A Review (2018, background): “Furthermore, evidence suggests that (i) the hippocampus supports additional forms of generalization through the recurrent interaction of episodic memories (Kumaran and McClelland, 2012) and (ii) if the new information is consistent with existing knowledge, then its integration into the neocortex is ”
Lifelong Learning of Spatiotemporal Representations With Dual-Memory Recurrent Self-Organization (2018, background): “…of memory consolidation where neocortical representations code for information acquired over more extended time periods than the hippocampus (e.g., Kumaran and McClelland, 2012; Kumaran et al., 2016), thereby yielding a higher degree of temporal slowness Temporal slowness results from the…”
Orthogonal Gradient Descent for Continual Learning (2019, background): “This class of methods also have their root in neuroscience (Kumaran and McClelland, 2012) making training samples as identically distributed as possible.”
Complementary learning systems within the hippocampus: A neural network modeling approach to reconciling episodic memory with statistical learning (2016, background): “In our model, the use of overlapping representations in CA1 would be expected to lead to catastrophic interference in blocked learning designs, whereas the orthogonalized representations used in REMERGE would be relatively unaffected.”
What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated. (2016, neutral): “While encoding-based overlap and retrieval-based models make divergent experimental predictions, empirical evidence to date does not de ﬁ nitely distinguish between them ([4,5,98] for discussion).”
Brain Mechanisms of Concept Learning (2019, background): “Although episodic inference can be achieved on-demand, from separate memories of individual events (Kumaran and McClelland, 2012), it can also result from memory integration, whereby new events are linked with prior related memories into a combined representation (Schlichting et al.”
Contextual Transformation Networks for Online Continual Learning (2021, background): “Moreover, the CLS theory also suggests that the interplay between the neocortex and the hippocampus attributes to the ability to recall knowledge and generalize to novel experiences (Kumaran & McClelland, 2012).”
MEMO: A Deep Network for Flexible Combination of Episodic Memories (2020, methodology): “This sequence of memories activation is reminiscent of the one predicted by computational model of the hippocampus (Kumaran & McClelland, 2012; Schapiro et al., 2017) and observed in neural data (Koster et al., 2018).”