Disentangling the Roles of Approach, Activation and Valence in Instrumental and Pavlovian Responding

By Quentin J. M. Huys, Roshan Cools, Martin Gölzer, Eva Friedel, Andreas Heinz, R. J. Dolan, Peter S. Dayan. PLoS Comput. Biol., 2011

443 citations recorded by the source corpus.

Hard-wired, Pavlovian, responses elicited by predictions of rewards and punishments exert significant benevolent and malevolent influences over instrumentally-appropriate actions. These influences come in two main groups, defined along anatomical, pharmacological, behavioural and functional lines. Investigations of the influences have so far concentrated on the groups as a whole; here we take the critical step of looking inside each group, using a detailed reinforcement learning model to distinguish effects to do with value, specific actions, and general activation or inhibition. We show a high degree of sophistication in Pavlovian influences, with appetitive Pavlovian stimuli specifically promoting approach and inhibiting withdrawal, and aversive Pavlovian stimuli promoting withdrawal and inhibiting approach. These influences account for differences in the instrumental performance of approach and withdrawal behaviours. Finally, although losses are as informative as gains, we find that subjects neglect losses in their instrumental learning. Our findings argue for a view of the Pavlovian system as a constraint or prior, facilitating learning by alleviating computational costs that come with increased flexibility.

Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/02221a64ef5425830b0390eff3cde1cba77866c8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cellular and Molecular Neuroscience

In Cognitive Neuroscience

Citations in the corpus

The successor representation in human reinforcement learning (2016, methodology): “For each participant, we estimated parameters 𝑏 and 𝑤 , by maximizing the likelihood of her revaluation scores, jointly with the group-level distributions over the entire population using an Expectation Maximization procedure 62 .”
Revealing Neurocomputational Mechanisms of Reinforcement Learning and Decision-Making With the hBayesDM Package (2016, background): “…and decision sciences further conﬁrmed the validity and usefulness of HBA and other hierarchical approaches (e.g., Ahn et al., 2014; Guitart-Masip et al., 2012; Huys et al., 2011; Katahira, 2016; Lee, 2011; Raja Beharelle, Polania, Hare, & Ruff, 2015; Shiffrin, Lee, Kim, & Wagenmakers, 2008).”
Go and no-go learning in reward and punishment: Interactions between affect and effect (2012, methodology): “These procedures are identical to those used by Huys et al. (2011), but we repeat them here for completeness.”
Bonsai Trees in Your Head: How the Pavlovian System Sculpts Goal-Directed Choices by Pruning Decision Trees (2012, methodology): “We have previously described our Bayesian model fitting and comparison approach [60], but repeat the description here for completeness.”
Model-Based Reasoning in Humans Becomes Automatic with Training (2015, methodology): “The model fitting routine follows that previously described by Huys and colleagues [48].”