The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis?

By Stanley E. Lazic. BMC Neuroscience, 2010

324 citations recorded by the source corpus.

BackgroundPseudoreplication occurs when observations are not statistically independent, but treated as if they are. This can occur when there are multiple observations on the same subjects, when samples are nested or hierarchically organised, or when measurements are correlated in time or space. Analysis of such data without taking these dependencies into account can lead to meaningless results, and examples can easily be found in the neuroscience literature.ResultsA single issue of Nature Neuroscience provided a number of examples and is used as a case study to highlight how pseudoreplication arises in neuroscientific studies, why the analyses in these papers are incorrect, and appropriate analytical methods are provided. 12% of papers had pseudoreplication and a further 36% were suspected of having pseudoreplication, but it was not possible to determine for certain because insufficient information was provided.ConclusionsPseudoreplication can undermine the conclusions of a statistical analysis, and it would be easier to detect if the sample size, degrees of freedom, the test statistic, and precise p-values are reported. This information should be a requirement for all publications.

Subfields: Neurology, Radiology, Nuclear Medicine and Imaging, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/cc72717c32a434b466a68fe9b3dd9b66189831d2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Neurology

In Radiology, Nuclear Medicine and Imaging

In Statistics and Probability

Citations in the corpus

Linear models and linear mixed effects models in R with linguistic applications (2013, unknown): “… – so much in fact, that there’s a whole literature associated with this violation, starting from Hurlbert (1984) for ecology, Freeberg and Lucas (2009) for psychology, Lazic (2010) for neuroscience and my own small paper for…”
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research* (2020, background): “Reporting n = 50 in this case is an example of pseudoreplication [26].”
Computation of measures of effect size for neuroscience data sets (2011, methodology): “…the documented existence of these misconceptions, the application of NHST suffers from frequent analysis mistakes (Garcı´a-Berthou & Alcaraz, 2004; Lazic, 2010; Nieuwenhuis et al. , 2011), and it has been suggested that replacing NHST with an analysis couched in the framework of MES and CIs is…”
Reporting animal research: Explanation and Elaboration for the ARRIVE guidelines 2019 (2019, methodology): “This may invalidate the analysis and resulting conclusions [21, 22] (see also item 7 – Statistical methods).”
Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0 (2020, background): “Reporting n = 50 in this case is an example of pseudoreplication [26].”
Experimental design and analysis and their reporting: new guidance for publication in BJP (2015, background): “This mistake is known as pseudo-replication (Lazic, 2010) and increases the chance of false positive results.”
If your P value looks too good to be true, it probably is: Communicating reproducibility and variability in cell biology (2019, methodology): “To convey anything about experimental reproducibility, P values and standard error of the mean should be calculated using independent measurements of a population of interest, typically observations from separate experiments (Lazic, 2010; Aarts et al., 2014; Naegle et al., 2015; Lazic et al., 2018).”