Identification of and Correction for Publication Bias

By Maximilian Kasy, Isaiah Andrews. The American Economic Review, 2017

342 citations recorded by the source corpus.

Some empirical results are more likely to be published than others. Selective publication leads to biased estimates and distorted inference. We propose two approaches for identifying the conditional probability of publication as a function of a study’s results, the first based on systematic replication studies and the second on meta-studies. For known conditional publication probabilities, we propose bias-corrected estimators and confidence sets. We apply our methods to recent replication studies in experimental economics and psychology, and to a meta-study on the effect of the minimum wage. When replication and meta-study data are available, we find similar results from both.(JEL C13, C90, I23, J23, J38, L82)

Subfields: Family Practice, History and Philosophy of Science, Safety Research

View Paper: https://www.semanticscholar.org/paper/bfc514cc0b64ef34a96d3c0248223c739b9930d1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Family Practice

In History and Philosophy of Science

In Safety Research

Citations in the corpus

Nudging for Tax Compliance: A Meta-Analysis (2019, positive): “, Andrews and Kasy 2019). Both directions of selection are plausible. If journals have preferences similar to those of researchers, the publication process may amplify the selection bias. On ther other hand, if the peer-review process effectively serve as a check against the behavior of researchers ”
INTERNATIONAL CORPORATE TAX AVOIDANCE: A REVIEW OF THE CHANNELS, MAGNITUDES, AND BLIND SPOTS (2018, positive): “general nonlinear (Andrews and Kasy, 2017), implying that the constants depicted in Table 5 are biased predictions of the true parameter.”
Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics (2020, methodology): “Our setting is well suited to applying the Andrews and Kasy (2019) measurement of publication bias.”
RCTs to Scale: Comprehensive Evidence from Two Nudge Units (2020, methodology): “5c-f we present funnel plots as in Andrews and Kasy (2019), plotting the point estimate and the standard errors, with bars indicating the results that are statistically significant.”
REPORTING GUIDELINES FOR META‐ANALYSIS IN ECONOMICS (2020, methodology): “In 2019, the American Economic Review published its first full paper that focuses on meta-analysis and publication bias (Andrews and Kasy, 2019).”
Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments (2019, background): “Third, there is well-documented publication bias towards significant findings (e.g., Franco et al., 2014; Andrews & Kasy, 2018; Christensen & Miguel, 2018; Abadie, 2020).”
Does economic globalisation affect income inequality? A meta‐analysis (2020, background): “Publication selection refers to a process in which results are chosen for their statistical significance (e.g. Brodeur et al. 2016; Andrews and Kasy 2019).”