Limitations of PET-PEESE and Other Meta-Analysis Methods

By T. Stanley. 2017

194 citations recorded by the source corpus.

A novel meta-regression method, precision-effect test and precision-effect estimate with standard errors (PET-PEESE), predicts and explains recent high-profile failures to replicate in psychology. The central purpose of this article is to identify the limitations of PET-PEESE for application to social/personality psychology. Using typical conditions found in social/personality research, our simulations identify three areas of concern. PET-PEESE performs poorly in research areas where there are only a few studies, all studies use small samples, and where there is very high heterogeneity of results from study to study. Nonetheless, the statistical properties of conventional meta-analysis approaches are much worse than PET-PEESE under these same conditions. Our simulations suggest alterations to conventional research practice and ways to moderate PET-PEESE weaknesses.

Subfields: Applied Psychology, Experimental and Cognitive Psychology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/62d9746bdeace29af787162583753de042a82d8a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Applied Psychology

1. Colin F. Camerer

In Experimental and Cognitive Psychology

1. Caroline F. Rowland
2. Colin F. Camerer
3. L. Wheeler
4. Giovanni Sala
5. Fernand R. Gobet

In Statistics and Probability

1. Michelle S. Peter
2. J. Lingwood
3. Julian M. Pine
4. Caroline F. Rowland
5. Colin F. Camerer
6. Claire L. Noble
7. Taisuke Imai
8. Tom Rutter

Citations in the corpus

• The impact of shared book reading on children's language skills: A meta-analysis (2018, background): “When the PET detects the presence of a non-zero effect at a 10% significance level (p < .100, one-tailed; Stanley, 2017), the PEESE estimate is usually preferred.”
• Meta-Analysis of Present-Bias Estimation using Convex Time Budgets (2019, background): “See Stanley and Doucouliagos (2012) and Stanley (2017) for in-depth discussion (especially on the limitations of these approaches).”
• A Social Comparison Theory Meta-Analysis 60+ Years On (2017, background): “The PET half of PET-PEESE was also not used because it works only when the heterogeneity is low and when there is no true underlying effect, neither of which were expected in our data (Stanley, 2017).”