Frontal Theta Links Prediction Errors to Behavioral Adaptation in Reinforcement Learning

By James F. Cavanagh, Michael J Frank, Theresa J. Klein, John J. B. Allen. NeuroImage, 2009

460 citations recorded by the source corpus.

Investigations into action monitoring have consistently detailed a fronto-central voltage deflection in the Event-Related Potential (ERP) following the presentation of negatively valenced feedback, sometimes termed the Feedback Related Negativity (FRN). The FRN has been proposed to reflect a neural response to prediction errors during reinforcement learning, yet the single trial relationship between neural activity and the quanta of expectation violation remains untested. Although ERP methods are not well suited to single trial analyses, the FRN has been associated with theta band oscillatory perturbations in the medial prefrontal cortex. Medio-frontal theta oscillations have been previously associated with expectation violation and behavioral adaptation and are well suited to single trial analysis. Here, we recorded EEG activity during a probabilistic reinforcement learning task and fit the performance data to an abstract computational model (Q-learning) for calculation of single-trial reward prediction errors. Single-trial theta oscillatory activities following feedback were investigated within the context of expectation (prediction error) and adaptation (subsequent reaction time change). Results indicate that interactive medial and lateral frontal theta activities reflect the degree of negative and positive reward prediction error in the service of behavioral adaptation. These different brain areas use prediction error calculations for different behavioral adaptations: with medial frontal theta reflecting the utilization of prediction errors for reaction time slowing (specifically following errors), but lateral frontal theta reflecting prediction errors leading to working memory-related reaction time speeding for the correct choice.

Subfields: Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/7cabc4203440cdbd0835b71f6dd1186d1416acb1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold (2011, background): “Specifically, theta-band power over the mPFC increases following punishment, error or conflict, and the degree of theta power increase predicts subsequent response time slowing, suggesting a direct role in adaptive contro”
How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis (2012, background): “, 2009), electroencephalography (Cavanagh et al., 2010) and genetic data (Frank et al.”
Great expectations: neural computations underlying the use of social norms in decision-making. (2011, background): “suggests that a neural signal, perhaps akin to a prediction error signal in basic reinforcement learning (Cavanagh et al., 2010), may be leveraged to calculate conflict between individual preferences and social norms (Klucharev et al.”
Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. (2012, result): “behavioral adaptation in response to changing outcomes on different time scales is consistent with existing theorizing and empirical data (Rushworth et al. 2002; Rushworth et al. 2003; Crone et al. 2006; Frank and Claus 2006; Hampton et al. 2006; Frank et al. 2007; Cavanagh et al. 2010).”
Theta activity and cognitive functioning: Integrating evidence from resting-state and task-related developmental electroencephalography (EEG) research (2024, background): “…their own behaviors, including response conflicts (Cohen and Cavanagh, 2011; Cohen and Donner, 2013; Nigbur et al., 2011), error monitoring (Cavanagh et al., 2009; Luu et al., 2004; Trujillo and Allen, 2007), task switching (Cunillera et al., 2012), and feedback processing (Cavanagh et al., 2010).”
Oscillatory signatures of monitoring and anticipatory strategies for probabilistic vs deterministic cues (2025, unknown): “Frontal theta oscillations have long been associated with a comprehensive mechanism governing the enactment of cognitive control ( Cavanagh & Frank, 2014), and they have been involved in goal-directed, strategic, and adaptive behavior ( Cavanagh et al., 2010; Cavanagh, Figueroa, et al., 2012).”
Neural oscillations guiding action during effects imagery. (2024, background): “These likely feedfor-ward predictive functions also fit current conceptions of TBA suggesting that the coding of predictions and prediction errors are central [71].”
The temporal dynamics of how the brain structures natural scenes. (2023, unknown): “TBA reﬂects prediction error signaling and the unexpectedness (surprise) of events (Cavanagh et al., 2010, 2012; Cavanagh & Frank, 2014).”