Struggles with survey weighting and regression modeling

By Andrew Gelman. 2007

635 citations recorded by the source corpus.

The general principles of Bayesian data analysis imply that models for survey responses should be constructed conditional on all variables that affect the probability of inclusion and nonresponse, which are also the variables used in survey weighting and clustering. However, such models can quickly become very complicated, with potentially thousands of poststratification cells. It is then a challenge to develop general families of multilevel probability models that yield reasonable Bayesian inferences. We discuss in the context of several ongoing public health and social surveys. This work is currently open-ended, and we conclude with thoughts on how research could proceed to solve these problems.

Subfields: Health, Sociology and Political Science, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/aa04a16df5300688b0d16d80a3f9832832385a1f

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Health

1. Justin T. Pickett
2. Eli Ben-Michael
3. Lawrence H. Cox
4. José L. Roldán
5. Thurasamy Ramayah
6. Andrew J. Thompson
7. Alan F. Karr
8. Xiao-Li Meng

In Sociology and Political Science

1. Thurasamy Ramayah
2. Hiram Ting
3. Justin T. Pickett
4. José L. Roldán
5. Eli Ben-Michael
6. Lawrence H. Cox
7. Andrew J. Thompson
8. Alan F. Karr

In Statistics and Probability

1. Douglas H. Johnson
2. Justin T. Pickett
3. Eli Ben-Michael
4. Lawrence H. Cox
5. José L. Roldán
6. Thurasamy Ramayah
7. Andrew J. Thompson
8. Alan F. Karr

Citations in the corpus

• Calibrated Bayes, an alternative inferential paradigm for official statistics (2012, neutral): “For discussions of model and design-based approaches to survey weights, see Little (2004, 2008) and Gelman (2007).”
• Sampling weight adjustments in partial least squares structural equation modeling: guidelines and illustrations (2020, methodology): “Hence, the construction and use of sampling weights should follow a rigorous process and fulfil the basic requirements and assumptions (e.g. Becker & Ismail, 2016; Cameron & Trivedi, 2005; Gelman, 2007; Shin, 2012).”
• Are Relational Inferences from Crowdsourced and Opt-in Samples Generalizable? Comparing Criminal Justice Attitudes in the GSS and Five Online Samples (2019, methodology): “Confounded sampling equates to conditional exchangeability, meaning that if all confounders are known, are measured without error, and have positivity in the sample, weighting on them would eliminate the bias (Gelman 2007; Mercer et al.”
• The Augmented Synthetic Control Method (2018, background): “McCullagh and Nelder (1989) and King and Zeng (2001) show that the MLE for logistic regression can be badly biased when the response is a rare event.”
• Risk‐Utility Paradigms for Statistical Disclosure Limitation: How to Think, But Not How to Act (2011, methodology): “Indeed, for regression models, a better approach is to incorporate variables affecting sampling design and nonresponse, if available, into the model rather than using weights (Fienberg, 2009; Gelman, 2007).”
• A plan for the North American Bat Monitoring Program (NABat) (2015, background): “, design weights) within an occupancy model (Gelman 2007).”
• Statistical paradises and paradoxes in big data (I): Law of large populations, big data paradox, and the 2016 US presidential election (2018, methodology): “Incidentally, the above derivation provides another justification of using weights as a covariate [e.g., Gelman (2007)], instead of using them to form the unstable Horvitz–Thompson estimate, as in Section 3.4.”