Environmental statistics and the trade-off between model-based and TD learning in humans

37 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Towards Continual Reinforcement Learning: A Review and Perspectives (2020, background): “It is believed that humans primarily rely on modelbased learning for dealing with tasks that have both high volatility and low noise (Simon and Daw, 2011).”
Reinforcement Learning and Causal Models (2017, background): “…the transition from model-based to model-free control can be understood in terms of a speed/accuracy tradeo↵ (Gershman, Horvitz, & Tenenbaum, 2015; Keramati, Dezfouli, & Piray, 2011). in which the reward and transition probabilities change quickly favor the model-free system (Simon & Daw, 2011).”
When Does Model-Based Control Pay Off? (2016, background): “For example, these changes might be too slow, such that model-free learning can adapt to these values at a rate that is proportional to the rate of change over time [53].”
Reinforcement learning, efficient coding, and the statistics of natural tasks (2015, background): “Just as task statistics will determine what form an efficient code will take, they also determine what class of planning or learning algorithm will be most efficient [70].”
Multiple Systems for Value Learning (2014, background): “Formal models extend this reasoning (Daw et al., 2005; Keramati et al., 2011; Simon and Daw, 2011a) to analyze under what circumstances the computational costs of model-based RL (for instance, in the brain, the opportunity cost of time and the caloric cost of firing neurons) are likely to be…”