Erroneous analyses of interactions in neuroscience: a problem of significance

By Sander Nieuwenhuis, Birte U. Forstmann, Eric-Jan Wagenmakers. Nature Neuroscience, 2011

834 citations recorded by the source corpus.

Subfields: Cognitive Neuroscience, Developmental and Educational Psychology

View Paper: https://www.semanticscholar.org/paper/6ea9a180f01c0c6a4b559511cf362bbfcfd73b87

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Developmental and Educational Psychology

Citations in the corpus

Moderate Levels of Activation Lead to Forgetting In the Think/No-Think Paradigm (2013, positive): “We are aware that a difference in significance values does not imply a significant difference (Nieuwenhuis et al., 2011).”
Sleeping on the motor engram: The multifaceted nature of sleep‐related motor memory consolidation (2017, positive): “Authors would erroneously conclude that a significant between-session improvement within the sleep group and a non-significant between-session change within the wake group was evidence for a sleep-dependent effect [see (Nieuwenhuis et al., 2011) for a discussion of this issue].”
cocor: A Comprehensive Solution for the Statistical Comparison of Correlations (2015, methodology): “To make a valid, meaningful, and interpretable comparison between two correlations, it is necessary to directly contrast the two correlations under investigation using an appropriate statistical test [7].”
Neuroimaging of the Periaqueductal Gray: State of the Field (2011, background): “Notably, the PAG neuroimaging literature reviewed here compose less than 10% of the entire PAG literature. al., 2004) or make erroneous statistical interpretations (Nieuwenhuis et al., 2011).”
Computation of measures of effect size for neuroscience data sets (2011, methodology): “…existence of these misconceptions, the application of NHST suffers from frequent analysis mistakes (Garcı´a-Berthou & Alcaraz, 2004; Lazic, 2010; Nieuwenhuis et al. , 2011), and it has been suggested that replacing NHST with an analysis couched in the framework of MES and CIs is more…”
Meaningful Mediation Analysis: Plausible Causal Inference and Informative Communication (2017, background): “Despite this, about half of 157 articles published in neuroscience claimed evidence for an interaction effect without reporting an explicit test for it (Nieuwenhuis et al. 2011).”
Statistical Significance and the Dichotomization of Evidence (2017, background): “…and Schindler (2016) as well as Thorstenson, Pazda and Elliot (2015), which was retracted for this error after being discussed on the blog), while Nieuwenhuis, Forstmann, and Wagenmakers (2011) documented that it is rife in neuroscience, appearing in half of neuroscience papers in top journals…”
A Generalized Form of Context-Dependent Psychophysiological Interactions (gPPI): A Comparison to Standard Approaches (2012, background): “…estimates between the three models are significantly different nor do we claim the values are different between models at the cluster level (Nieuwenhuis et al., 2011); however, since the goal is to identify which method is optimal at detecting psychophysiological interactions, their…”
Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking (2016, background): “Most of the researcher DFs have been described in previous publications (Kriegeskorte et al., 2009; Nieuwenhuis et al., 2011; Simmons et al., 2011; Wagenmakers et al., 2011; Bakker et al., 2012; John et al., 2012; Chambers, 2013; Francis, 2013; Bakker and Wicherts, 2014; Simonsohn et al., 2014; Stee”