One and Done? Optimal Decisions From Very Few Samples

By E. Vul, Noah D. Goodman, T. Griffiths, Josh Tenenbaum. 2014

463 citations recorded by the source corpus.

In many learning or inference tasks human behavior approximates that of a Bayesian ideal observer, suggesting that, at some level, cognition can be described as Bayesian inference. However, a number of findings have highlighted an intriguing mismatch between human behavior and standard assumptions about optimality: People often appear to make decisions based on just one or a few samples from the appropriate posterior probability distribution, rather than using the full distribution. Although sampling-based approximations are a common way to implement Bayesian inference, the very limited numbers of samples often used by humans seem insufficient to approximate the required probability distributions very accurately. Here, we consider this discrepancy in the broader framework of statistical decision theory, and ask: If people are making decisions based on samples--but as samples are costly--how many samples should people use to optimize their total expected or worst-case reward over a large number of decisions? We find that under reasonable assumptions about the time costs of sampling, making many quick but locally suboptimal decisions based on very few samples may be the globally optimal strategy over long periods. These results help to reconcile a large body of work showing sampling-based or probability matching behavior with the hypothesis that human cognition can be understood in Bayesian terms, and they suggest promising future directions for studies of resource-constrained cognition.

Subfields: Cognitive Neuroscience, General Decision Sciences

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In General Decision Sciences

Citations in the corpus

Troubles with Bayesianism: An introduction to the psychological immune system (2018, background): “In fact, it is recently been argued that mere sampling from the posterior is almost as optimal (for decision making) as using the full posterior, even when one just takes a single sample from the posterior (and often it looks like single samples are themselves pragmatically ideal; Vul et al., 2014).”
Unpacking the Exploration–Exploitation Tradeoff: A Synthesis of Human and Animal Literatures (2015, background): “optimal stopping (Seale & Rapoport, 2000), mutual mate choice (Todd & Miller, 1999), Bayesian optional stopping models (Edwards, 1965), Bayesian observer models (Vul et al., 2009), the accumulation of evidence to a threshold criterion (Busemeyer & Townsend, 1993; Ratcliff, 1978; Vickers, 1979), and ”
Speed, Accuracy, and the Optimal Timing of Choices (2018, background): “Vul et al. (2014) studies the optimal predetermined sample size for an agent whose cost of time arises from the opportunity to make future decisions; they find that for a range of parameters the optimal sample size is one.”
Perception, Action, and Utility: The Tangled Skein (2012, background): “Vul et al. (2009) analyzed the effects of imposing sampling costs on an such an agent.”
Multistability and Perceptual Inference (2012, background): “An emerging literature suggests that Monte Carlo approximations might be employed in human (Sanborn et al., 2006; Levy et al., 2009; Vul et al., 2009) and animal (Daw and Courville, 2007) cognition, and several authors have proposed that the spiking activity of neurons might be interpreted as a set ”
The Human Kernel (2015, background): “We assume the predictions y∗ are samples from the learner’s posterior distribution over possible functions, following results showing that human inferences and judgments resemble posterior samples across a wide range of perceptual and decision-making tasks [27, 28, 29].”
Interactive Activation and Mutual Constraint Satisfaction in Perception and Cognition (2014, background): “Researchers coming from a computational-level starting point have begun to consider the importance of this issue (Norris, 2013; Vul, Goodman, Griffiths, & Tenenbaum, 2014).”
Simulation as an engine of physical scene understanding (2013, background): “The IPE may dramatically simplify objects’ geometry, mass density distributions, and physical interactions, relative to what the ODE allows; and instead of running many Monte Carlo simulations, the IPE may encode probabilities very coarsely by using only one or a few samples (as people do in simpler”