Modeling Clustered Data with Very Few Clusters

By Daniel M. McNeish, Laura M. Stapleton. Multivariate Behavioral Research, 2016

372 citations recorded by the source corpus.

ABSTRACT Small-sample inference with clustered data has received increased attention recently in the methodological literature, with several simulation studies being presented on the small-sample behavior of many methods. However, nearly all previous studies focus on a single class of methods (e.g., only multilevel models, only corrections to sandwich estimators), and the differential performance of various methods that can be implemented to accommodate clustered data with very few clusters is largely unknown, potentially due to the rigid disciplinary preferences. Furthermore, a majority of these studies focus on scenarios with 15 or more clusters and feature unrealistically simple data-generation models with very few predictors. This article, motivated by an applied educational psychology cluster randomized trial, presents a simulation study that simultaneously addresses the extreme small sample and differential performance (estimation bias, Type I error rates, and relative power) of 12 methods to account for clustered data with a model that features a more realistic number of predictors. The motivating data are then modeled with each method, and results are compared. Results show that generalized estimating equations perform poorly; the choice of Bayesian prior distributions affects performance; and fixed effect models perform quite well. Limitations and implications for applications are also discussed.

Subfields: Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/217ae3922a6f771cc7de4561c095fa78aee40d7c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

Citations in the corpus

Small Sample Size Solutions (2020, methodology): “However, if the 222 Joop Hox and Daniel McNeish interest is only in the fixed effects of the lowest-level variables, these approaches are simple and effective with as few as four clusters (McNeish & Stapleton, 2016b).”
Multilevel Modeling and Ordinary Least Squares Regression: How Comparable Are They? (2018, methodology): “In addition, we applied a Kenward-Roger (KR, 1997) correction to adjust standard errors, which has shown to result in less biased standard error estimates with a low number of clusters (Bell, Morgan, Schoeneberger, Kromrey, & Ferron, 2014; McNeish & Stapleton, 2016).”
On Using Bayesian Methods to Address Small Sample Problems (2016, methodology): “…deteriorate if the model becomes more complex or sample size is even further reduced; Ferron, Bell, Hess, Rendina-Gobioff, & Hibbard, 2009; McNeish & Stapleton, 2016).3 The weakly informative prior showed some downward bias at smaller sample sizes but entered into the reasonable range…”
Analyzing Cross-Sectionally Clustered Data Using Generalized Estimating Equations (2021, methodology): “…2001, p. 178) and that the actual algorithm of GEEs (e.g., compared to ordinary least squares [OLS] regression) can be rather “opaque” (i.e., not accessible) for applied researchers, requiring knowledge of partial derivatives and matrix algebra to understand (McNeish & Stapleton, 2016, p. 500).”