How persuasive is a good fit? A comment on theory testing.

By S. Roberts, Harold Pashler. Psychology Review, 2000

827 citations recorded by the source corpus.

Subfields: Artificial Intelligence, History and Philosophy of Science, Sociology and Political Science

View Paper: https://www.semanticscholar.org/paper/3687954c6fff34b2e74fedc3b62d713daf971f0a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In History and Philosophy of Science

In Sociology and Political Science

Citations in the corpus

A discounting framework for choice with delayed and probabilistic rewards. (2004, background): “[PubMed: 10789200] Rodgers JL, Rowe DC. Theory development should begin (but not end) with good empirical fits: A comment on Roberts and Pashler (2000).”
Executive functions and the generation of “random” sequential responses: A computational account (2016, background): “A more detailed examination of the model’s parameter space is in order both to understand the mechanisms within the model that lead to deviations from expected values for the various measures and to ensure that the model cannot capture any pattern of behaviour (cf. Roberts & Pashler, 2000).”
What mediation analysis can (not) do (2011, neutral): “This is a matter of clever theorizing, convergent validation, analytical conventions, and experimental tests of antecedence–consequence relations (Roberts & Pashler, 2000; Campbell & Fiske, 1959; Slovic, 1962).”
Investigating Determinants of Software Developers' Intentions to Follow Methodologies (2003, background): “Although some theorists regard goodness of fit to be overrated as evidence of a theory’s validity [53], the presence of incorrect structural assumptions in a proposed model can distort estimates of those parameters within a model that are correctly specified [16].”
Structural equation modeling: strengths, limitations, and misconceptions. (2005, background): “It reflects the more general absence of point or range predictions in clinical psychology and related disciplines (e.g., Meehl 1978, Roberts & Pashler 2000).”
Homo Heuristicus: Why Biased Minds Make Better Inferences (2009, background): “Despite this, Roberts and Pashler (2000) estimated that, in psychology alone, the number of articles relying on a good fit as the only indication of a good model runs into the thousands.”
Striking a Blow for Sanity in Theories of Rationality (2004, background): “However, the belief that a good fi t between model and data would provide empirical evidence for the model is unfounded if the model has numerous free parameters (Roberts & Pashler, 2000).”
Marketing innovation: A consequence of competitiveness (2016, background): “Furthermore, unfortunately, not many studies examine and report on predictive validity, almost all studies test and report only on fit appropriateness (Roberts & Pashler, 2000).”
The Priority Heuristic: Making Choices Without Trade-Offs (2006, background): “Models with smaller numbers of adjustable parameters, which embody Occam’s razor, have a higher posterior probability in Bayesian model comparison (MacKay, 1995; Roberts & Pashler, 2000 ).”
The null hypothesis significance-testing debate and its implications for personality research. (2007, background): “…several theoretical models, formalizing them, and exploring their unique implications for the data, it is possible to understand the constraints of a model better and to identify exactly which kinds of data are necessary to test alternative theoretical assumptions (see Roberts & Pashler, 2000).”
INTERPERSONAL RELATIONS AND GROUP PROCESSES Patterns of Stability in Adult Attachment: An Empirical Test of Two Models of Continuity and Change (2011, neutral): “Thus, relaxing this constraint does not provide a better test of the theoretical principles per se, even if doing so produces better fit statistics (see S. Roberts & Pashler, 2000).”
Toward a method of selecting among computational models of cognition. (2002, methodology): “Just because a model fits data well does not necessarily imply that the regularity one seeks to capture in the data is well approximated by the model (Roberts & Pashler, 2000).”