The Spread of Evidence-Poor Medicine via Flawed Social-Network Analysis

By Russell Lyons. 2010

204 citations recorded by the source corpus.

The chronic widespread misuse of statistics is usually inadvertent, not intentional. We find cautionary examples in a series of recent papers by Christakis and Fowler that advance statistical arguments for the transmission via social networks of various personal characteristics, including obesity, smoking cessation, happiness, and loneliness. Those papers also assert that such influence extends to three degrees of separation in social networks. We shall show that these conclusions do not follow from Christakis and Fowler's statistical analyses. In fact, their studies even provide some evidence against the existence of such transmission. The errors that we expose arose, in part, because the assumptions behind the statistical procedures used were insufficiently examined, not only by the authors, but also by the reviewers. Our examples are instructive because the practitioners are highly reputed, their results have received enormous popular attention, and the journals that published their studies are among the most respected in the world. An educational bonus emerges from the difficulty we report in getting our critique published. We discuss the relevance of this episode to understanding statistical literacy and the role of scientific review, as well as to reforming statistics education.

Subfields: Experimental and Cognitive Psychology, Health, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Experimental and Cognitive Psychology

In Health

In Statistics and Probability

Citations in the corpus

Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019, background): “For instance, network studies of peer inﬂuence and social contagion suﬀer from a stubborn challenge of disambiguating such eﬀects from homophily among peers and within communities (Christakis and Fowler, 2007; Lyons, 2011; Shalizi and Thomas, 2011).”
The Book of Why: The New Science of Cause and Effect† (2019, background): “It is also difﬁcult to comprehend its nuanced refuta-tions by Lyons (2011) and by Shalizi and Thomas (2011).”
Social Networks and Causal Inference (2013, methodology): “Unfortunately, as Lyons (2011) notes, this is not the only source of dependence in the data.”
Systems Science Methods in Public Health (2012, unknown): “The methods and conclusions of some of these studies have been challenged (34, 95), but the visibility of this work has helped to highlight the continuing importance of NA methods in modern public health science.”
Homophily and Contagion Are Generically Confounded in Observational Social Network Studies (2010, background): “Treated at face value, this can produce an incoherent probability distribution for the evolution of the system (Lyons, 2010), as well as implying a scarcely-comprehensible notion of simultaneous causation (rather than coupled behavior or feedback); this can be somewhat salvaged by considering it as…”
Handbook of Big Data (2016, background): “Stepping up and down the ladder is achieved by repeatedly hitting • > to lower the threshold and obtain greater protection • to raise the threshold and lose protection.”