EDA for HLM: Visualization when Probabilistic Inference Fails

By Jake Bowers, Katherine W. Drake. Political Analysis, 2005

97 citations recorded by the source corpus.

Nearly all hierarchical linear models presented to political science audiences are estimated using maximum likelihood under a repeated sampling interpretation of the results of hypothesis tests. Maximum likelihood estimators have excellent asymptotic properties but less than ideal small sample properties. Multilevel models common in political science have relatively large samples of units like individuals nested within relatively small samples of units like countries. Often these level-2 samples will be so small as to make inference about level-2 effects uninterpretable in the likelihood framework from which they were estimated. When analysts do not have enough data to make a compelling argument for repeated sampling based probabilistic inference, we show how visualization can be a useful way of allowing scientific progress to continue despite lack of fit between research design and asymptotic properties of maximum likelihood estimators. Somewhere along the line in the teaching of statistics in the social sciences, the importance of good judgment got lost amid the minutiae of null hypothesis testing. It is all right, indeed essential, to argue flexibly and in detail for a particular case when you use statistics. Data analysis should not be pointlessly formal. It should make an interesting claim; it should tell a story that an informed audience will care about, and it should do so by intelligent interpretation of appropriate evidence from empirical measurements or observations. —Abelson, 1995, p. 2 With neither prior mathematical theory nor intensive prior investigation of the data, throwing half a dozen or more exogenous variables into a regression, probit, or novel maximum-likelihood estimator is pointless. No one knows how they are interrelated, and the high-dimensional parameter space will generate a shimmering pseudo-fit like a bright coat of paint on a boat's rotting hull. —Achen, 1999, p. 26

Subfields: Political Science and International Relations, Sociology and Political Science, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/bcb61d004a702c22d28961521c665143c0d5d251

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Political Science and International Relations

In Sociology and Political Science

In Statistics and Probability

Citations in the corpus

Regression Analysis of Country Effects Using Multilevel Data: A Cautionary Tale (2013, background): “See Bowers and Drake (2005) and the empirical illustration in Section 8 for examples.”
Multilevel Modelling of Country Effects: A Cautionary Tale (2016, methodology): “On this, see Bowers and Drake (2005) and also the examples in section 8 of this article’s Supplementary Material. The twostep approach may reveal features of the data, such as country groupings, that are worthy of further investigation; or could highlight outliers that have an undue influence owing ”
Empirical Strategies for Various Manifestations of Multilevel Data (2005, background): “Follow Bowers and Drake (2005) to consider this general expression of a hierarchical linear model (HLM), with one zj and xij: yij ¼ b0j þ b1jxij þ eij; ð1Þ b0j ¼ c00 þ c01zj þ u0j; ð2Þ b1j ¼ c10 þ c11zj þ u1j; ð3Þ ) yij ¼ c00 þ c01zj þ u0j þ ðc10 þ c11zj þ u1jÞxij þ eij; ð4Þ ) yij ¼ c00 þ c01zj þ…”