Hippocampal Contributions to Control: The Third Way

By Máté Lengyel, Peter S. Dayan. 2007

256 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

RUDDER: Return Decomposition for Delayed Rewards (2018, neutral): “starting time, length, and the action selection strategy with safety constraints. (II) Lessons replay buffer. The lessons replay buffer is an episodic memory, which has been used for episodic control [62] and for episodic backward update to efﬁciently propagate delayed rewards [61]. If RUDDER safe e”
Self-Imitation Learning (2018, background): “Though the role of exploitation for exploration has been discussed (Thrun, 1992 Episodic control Episodic control (Lengyel & Dayan, 2008) can be viewed as an extreme way of exploiting past experiences in the sense that the agent repeats the same actions that gave the best outcome in the past.”
Replay in Deep Learning: Current Approaches and Missing Biological Elements (2021, positive): “For example (Pritzel et al., 2017; Lengyel & Dayan, 2008; Blundell et al., 2016) take inspiration from the role the hippocampus plays in making decisions to develop agents that learn much faster than other approaches.”
Model-Free Episodic Control (2016, background): “This prediction agrees with those of [5, 16] with respect to the statistical trade-offs between systems.”
Neural Episodic Control (2017, background): “Deep Q-Network agents (DQN; Mnih et al., 2015) use Qlearning (Watkins & Dayan, 1992) to learn a value function Q(st, at) to rank which action at is best to take in each state st at step t.”
Training Agents using Upside-Down Reinforcement Learning (2019, background): “Episodic Control [Lengyel and Dayan, 2008; Blundell et al., 2016] is philosophically similar to our work in taking an alternate perspective on learning to act by directly associating good actions to high returns.”
Integrating memories to guide decisions (2015, background): “It remains to be understood to what extent the decision phenomena we have examined are supported by one or the other or both of these underlying representations [61].”
A framework for studying the neurobiology of value-based decision making (2008, background): “Second, are there more than three valuation systems? Lengyel and Daya”
Goals and Habits in the Brain (2013, unknown): “In fact, this has a bearing on a suggestion that there is another form of controller, an ‘‘episodic controller,’’ that involves these very structures (Lengyel and Dayan, 2008).”
Prioritized memory access explains planning and hippocampal replay (2017, background): “An important goal of future work will be to tease apart the role of episodic vs. semantic knowledge in computing action values, and understand their relative prioritization (Gershman and Daw, 2017; Lengyel and Dayan, 2008).”
Sample-Efficient Deep Reinforcement Learning via Episodic Backward Update (2018, methodology): “Such episodic control method is how humans normally recognize the cause and effect relationship (Lengyel & Dayan, 2007).”
On the link between conscious function and general intelligence in humans and machines (2022, background): “Initial attempts to imitate hippocampal episodic control (Lengyel & Dayan, 2007) in AI agents used non-parametric (Blundell et al., 2016) and semi-parametric (Pritzel et al., 2017) memories (which store state information from every timestep), enabling rapid learning when compared to standard deep…”
How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis (2012, methodology): “Some of these works also propose arbitrations between different parallel ways of using model information (e.g. see Lengyel & Dayan, 2007 for a three-controller system).”
What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated. (2016, unknown): “…perspectives that emphasize the role of the human hippocampus in prospection [77], imagination [78,79], and the potential utility of episodic control of behavior over control based on learned summary statistics in some circumstances (e.g., given relatively little experience in an environment) [80].”
Reinforcement Learning, Fast and Slow. (2019, unknown): “Intriguingly, recent work on reinforcement learning in animals and humans has increasingly emphasized the potential contribution of episodic memory, with evidence accruing that estimates of state and action value are based on retrieved memories for specific past action-outcome observations [41, 42, ”
The successor representation in human reinforcement learning (2016, background): “Such computation may be intractable (requiring error-prone approximations) in large search spaces such as wide and deep trees 9,10 .”
Decision theory, reinforcement learning, and the brain (2008, background): “Other structures may also be involved, for instance with the hippocampus contributing to control based on episodic memory (Lengyel and Dayan, 2008).”