Reinforcement learning: the good, the bad and the ugly.

By Peter S. Dayan, Yael Niv. 2008

686 citations recorded by the source corpus.

Subfields: Cognitive Neuroscience, Experimental and Cognitive Psychology, General Decision Sciences

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Experimental and Cognitive Psychology

In General Decision Sciences

Citations in the corpus

Human Neuroscience (neutral): “These difficulties have been avoided by descriptive approaches to learning, which are not grounded in probability theory, notably some forms of reinforcement learning (RL), where agents learn the " value " of different stimuli and actions (Sutton and Barto, 1998; Dayan and Niv, 2008).”
An Introduction to Deep Reinforcement Learning (2018, neutral): “…neuroscience and, as such, the reader is referred to the work of Sutton and Barto (2017), Niv (2009), Lee et al. (2012), Holroyd and Coles (2002), Dayan and Niv (2008), Dayan and Daw (2008), Montague (2013), and Niv and Montague (2009) for an in-depth history of the development of reinforcement…”
The ecology of human fear: survival optimization and the nervous system (2015, background): “However, these and other derivative models fail to predict outcomes in changing contexts and circumstances, notably exploratory behavior, novelty, uncertainty, and extinction (Miller et al., 1995; Phelps and LeDoux, 2005; Dayan and Niv, 2008).”
Hierarchical Reinforcement Learning: A Survey (2015, background): “According to these changes, the agent obtains rewards based on the taken actions [1], [2].”
Human Reinforcement Learning Subdivides Structured Action Spaces by Learning Effector-Specific Values (2009, background): “This is a vivid problem for prominent accounts of the brain’s mechanisms for reinforcement learning (RL) (Daw and Doya, 2006; Dayan and Niv, 2008), which envision that the brain learns to map each candidate action to its expected consequences, to choose the best one.”
Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates. (2012, positive): “In building a formal theory, we take as our point of departure an insight recently expressed by Daw et al. (2005; see also Dayan & Niv, 2008), which is that goal-directed decision making can be viewed as a version of model-based reinforcement learning.”
Behavioral / Systems / Cognitive Striatal Prediction Error Modulates Cortical Coupling (2010, neutral): “Clearly, there are numerous potential models of such a process (Dayan and Daw, 2008; Dayan and Niv, 2008).”
Model-Based Reasoning in Humans Becomes Automatic with Training (2015, background): “In Daw and colleagues' framework, retrospective learning is accomplished with model-free strategies in which rewarded actions tend to be repeated, but the underlying structure of the world that gives rise to these rewards is not learned [6] [7].”
Model-based predictions for dopamine (2017, neutral): “However, when a reward is delivered unexpectedly early, dopamine neurons do not display a phasic decrease in activity at the original expected time of reward, as would be implied by the CSC, in which a prediction error updates the value of the current, and not subsequent, timepoint states [18,19].”
Process Account of Curiosity and Interest: A Reward-Learning Perspective (2018, methodology): “The framework is based on reward-learning (or reinforcement-learning) models (Dayan and Niv 2008; Montague and Berns 2002), but we incorporated several features that are specific to the knowledge-acquisition process (e.g., new knowledge provides room for further knowledge gaps).”
Teaching with Rewards and Punishments: Reinforcement or Communication? (2015, background): “Keywords: pedagogy; reward; punishment; reinforcement learning; feedback; evaluative feedback; communication”
Neural basis of reinforcement learning and decision making. (2012, background): “Since this topic has been frequently reviewed (Dayan & Niv 2008, van der Meer & Redish 2011, Ito & Doya 2011, Bornstein & Daw, 2011), we only briefly summarize the essential elements of reinforcement learning theory and focus on the following questions.”
Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli (2011, positive): “These findings led to the influential hypothesis that phasic DA cell activity encodes a reward prediction error, which is critical for reinforcement-dependent learning (Schultz, 1998, 2007, 2010; D'Ardenne et al., 2008; Dayan and Niv, 2008).”