NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript NIH Public Access Author Manuscript Neuron. Author manuscript.

2011

Citations in the corpus

• Rule abstraction, model-based choice, and cognitive reflection (2016, methodology): “Typical experiments using this task find contributions of both model-based and model-free strategies, both within individuals and at the population level (Daw et al., 2011).”
• Simple Plans or Sophisticated Habits? State, Transition and Learning Interactions in the Two-Step Task (2015, methodology): “The two-step task [7] represents one recently popular approach to creating such a task, attracting a substantial number of human studies [8,11,12,31–43].”
• Neural computations underlying arbitration between model-based and model-free learning (2013, background): “Empirical evidence for this computational distinction has emerged in recent years (Daw et al., 2011; Gläscher et al., 2010; Wunderlich et al., 2012).”
• Model-Based Reasoning in Humans Becomes Automatic with Training (2015, methodology): “In keeping with other studies utilizing the two-step task [16,18,19,21], we repeated the regression analysis but now only considering the influence of events occurring on the immediately preceding trial.”
• Prioritized memory access explains planning and hippocampal replay (2017, unknown): “…of the substrates of these two sorts of learning would explain results (puzzling on a separate systems view) that dopaminergic activity is both informed by (Daw et al., 2011; Sadacca et al., 2016) and supports (Deserno et al., 2015; Doll et al., 2016; Sharpe et al., 2017) model-based evaluation.”
• Learning to build physical structures better over time (2020, background): “…behavior is potentially limited by the historically narrow focus on tasks with relatively low state-space complexity (van Opheusden, Galbiati, Bnaya, Li, & Ma, 2017), as well as abstract action spaces and state transitions (Daw et al., 2011) that are far removed from the physical environment.”
• Disorders of compulsivity: a common bias towards learning habits (2014, unknown): “In particular, we assessed the relationship between model-free or model-based biases and grey matter volume specifically focusing on regions identified in previous studies including striatal regions, orbitofrontal, lateral prefrontal and parietal cortex (4, 9, 10).”
• Prefrontal cortex as a meta-reinforcement learning system (2018, methodology): “Each agent generated a reward prediction error on each step, as described by Daw et al.(15), but with the expectation of reward at the second stage being p(H) × 0.”
• Psychology of Habit. (2016, background): “Recent dual-system RL models propose that response selection is based on action values represented in ventromedial prefrontal cortex (vmPFC) that are jointly determined by a model-free (habit) and prospective planning controllers (e.g., Daw et al. 2011).”
• Multiple Systems for Value Learning (2014, result): “Also, studies examining whether expected value correlates in this region comply more with model-based (versus model-free) values, as predicted by computational models, have repeatedly shown evidence for modelbased values there (Beierholm et al., 2011; Daw et al., 2011; Hampton et al., 2006, 2008) (F”
• How Prediction Errors Shape Perception, Attention, and Motivation (2012, background): “, 2008; Franklin and Wolpert, 2011), language (Kutas and Hillyard, 1980), memory (Erickson and Desimone, 1999; Kumaran and Maguire, 2006, 2009), cognitive control (Alexander and Brown, 2011), and motivational value processing (Schultz, 1998; Hare et al., 2008; Daw et al., 2011).”
• Integrating memories to guide decisions (2015, background): “Similarly, neuroscientific data — including pretraining lesions [11] or neural correlates of the final, integrated decision variables [10] — do not speak to this question.”
• The Role of Executive Function in Shaping Reinforcement Learning. (2020, background): “Similar results are observed in planning tasks, where an EF-dependent planned expectation of reward modulates the classic representation of RPEs in the striatum [22].”
• Ten simple rules for the computational modeling of behavioral data (2019, methodology): “…advanced modeling techniques should consult the many excellent tutorials, didactic examples, and books on the topic (Busemeyer and Diederich, 2010; Daw, 2011; Daw and Tobler, 2014; Heathcote et al., 2015; Huys, 2017; Turner et al., 2013; Vandekerckhove et al., 2015; Wagenmakers and Farrell, 2004;…”