Learning Structures: Predictive Representations, Replay, and Generalization
By Ida Momennejad. 2020
150 citations recorded by the source corpus.
Memory and planning rely on learning the structure of relationships among experiences. Compact representations of these structures guide flexible behavior in humans and animals. A century after ‘latent learning’ experiments summarized by Tolman, the larger puzzle of cognitive maps remains elusive: how does the brain learn and generalize relational structures? This review focuses on a reinforcement learning (RL) approach to learning compact representations of the structure of states. We review evidence showing that capturing structures as predictive representations updated via replay offers a neurally plausible account of human behavior and the neural representations of predictive cognitive maps. We highlight multi-scale successor representations, prioritized replay, and policy-dependence. These advances call for new directions in studying the entanglement of learning and memory with prediction and planning.
Subfields: Molecular Biology
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Molecular Biology
- Neil Burgess
- Maxim Bazhenov
- Doina Precup
- Terrence J. Sejnowski
- Irina Rish
- Timothy Edward John Behrens
- Hava T. Siegelmann
- Tyler L. Hayes
Citations in the corpus
- Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, positive): “…further suggests that selective and partial replay in tasks involving reward allow humans to perform generalization and model-based reinforcement learning (Momennejad, 2020), which has inspired several algorithms in machine learning (see Cazé, Khamassi, Aubin, & Girard, 2018, for a review).”
- Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “Additionally, there is strong evidence in the literature of hippocampal replay facilitating the model-based planning process for human RL (Mattar and Daw, 2018; Momennejad, Otto, Daw, and Norman, 2018; Vikbladh, Meager, King, Blackmon, Devinsky, Shohamy, Burgess, and Daw, 2019; Momennejad, 2020).”
- The Tolman-Eichenbaum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal Formation (2019, background): “, 2017) to propose that entorhinal and frontal cortices provide bases for successor features at various hierarchical scales; more formally, entorhinal cortex is proposed to represent the eigenspace of successor features (Momennejad, 2020).”