Reinforcement learning in the brain

By Yael Niv. 2009

788 citations recorded by the source corpus.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Molecular Biology, Neuroscience

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

In Molecular Biology

In Neuroscience

Citations in the corpus

What can the brain teach us about building artificial intelligence? (2016, background): “ows for fast selection of actions. There is indeed substantial evidence that the brain uses similar model-free learning algorithms in simple associative learning or discrimination learning tasks (see Niv, 2009, for a review). In particular, the phasic ring of midbrain dopaminergic neurons is qualita”
Recurrent Environment Simulators (2017, background): “…for perception (O’Regan & Noë, 2001); in neuroscience, environment simulation forms part of deliberative planning systems used by the brain (Niv, 2009); and in reinforcement learning, the ability to imagine the future evolution of an environment is needed to form predictive state…”
Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “For example, Hassabis, Kumaran, Summerfield, and Botvinick (2017) provides a survey on neuroscience inspired AI, and Niv (2009) details evidence that RL happens in the human brain.”
Multi-agent Reinforcement Learning in Sequential Social Dilemmas (2017, background): “Recall also that DQN is in the class of reinforcement learning algorithms that is generally considered to be the leading candidate theory of animal habit-learning [50, 42].”
Reinforcement Learning and Causal Models (2017, background): “Although it is still a matter of debate what particular form of TD learning is used by the brain (Niv, 2009), all TD algorithms embody the basic prediction error logic laid out above.”
An Introduction to Deep Reinforcement Learning (2018, neutral): “…and rich literature about the connections between RL and neuroscience and, as such, the reader is referred to the work of Sutton and Barto (2017), Niv (2009), Lee et al. (2012), Holroyd and Coles (2002), Dayan and Niv (2008), Dayan and Daw (2008), Montague (2013), and Niv and Montague (2009) for…”
Error and expectation in language learning: The curious absence of mouses in adult speech (2013, background): “…that make assumptions similar to those just described have been successfully applied to the study of decision making, executive function, habitual learning, and response selection in humans (McClure 2003, Montague et al. 2004, Montague et al. 1996, Niv 2009, Schultz 1998, 2006, Waelti et al. 2001).”
The mysterious motivational functions of mesolimbic dopamine. (2012, positive): “This pattern of activity in putative DA neurons has provided a formal theoretical basis for the involvement of fast phasic DA signaling in reinforcement learning models (Schultz et al., 1997; Bayer and Glimcher, 2005; Niv, 2009; Schultz, 2010).”
On the link between conscious function and general intelligence in humans and machines (2022, positive): “While model-free RL, incorporating policies and/or value functions, is a core component of decision making in biological agents (Niv, 2009), and has even achieved superhuman results in virtual domains (Berner et al.”
Perception, Action, and Utility: The Tangled Skein (2012, background): “…according to their probability given the state.3 Much work concerns incremental learning rules for taking a running average over experienced outcomes’ utilities, so as to learn an estimate of their expectation; this is the purview of reinforcement learning (see Niv, 2009, for a recent review).”
Goals and Habits in the Brain (2013, neutral): “Many of the issues that we lack space to discuss are treated by others (Rangel et al., 2008; Botvinick, 2012; Berridge, 2001; Padoa-Schioppa and Assad, 2006; Daw et al., 2006a; Dayan and Daw, 2008; Balleine and O’Doherty, 2010; Yin and Knowlton, 2006; Maia, 2009; Niv, 2009; Doll et al., 2012).”
Evaluating the TD model of classical conditioning (2012, background): “The TD model of conditioning has become the leading explanation for conditioning in neuroscience, due to the correspondence between the phasic firing of dopamine neurons and the reward-prediction error that drives learning in the model (Schultz, Dayan, & Montague, 1997; for reviews, see Ludvig, Bell”
The minority report: some common assumptions to reconsider in the modelling of the brain and behaviour (2016, background): “…basal ganglia in terms of reward-and prediction-related switching of the so-called direct and indirect pathways that channel the corticostriatal circuits (Botvinick et al., 2009; Cohen & Frank, 2009; Niv, 2009; O’Reilly, Noelle, Braver, & Cohen, 2002; O’Reilly & Frank, 2006; Redgrave et al., 2011).”
Toward a Brain-Inspired System: Deep Recurrent Reinforcement Learning for a Simulated Self-Driving Agent (2019, background): “Niv (2009) categorized major learning components into four classes: the neocortex, the hippocampal formation (explicit memory storage system), the cerebellum (adaptive control system), and the basal ganglia (reinforcement learning).”
Motivation and cognitive control: from behavior to neural mechanism. (2015, background): “…of reward and reward-based decision are represented or processed within and across each of these structures, with important distinctions holding between reward and expected-value signals and between representations of value and prediction-error signals related to reinforcement learning (Niv 2009).”