External Validity: From Do-Calculus to Transportability Across Populations

By Judea Pearl, Elias Bareinboim. 2014

379 citations recorded by the source corpus.

The generalizability of empirical findings to new environments, settings or populations, often called "external validity," is essential in most scientific explorations. This paper treats a particular problem of generalizability, called "transportability," defined as a license to transfer causal effects learned in experimental studies to a new population, in which only observational studies can be conducted. We introduce a formal representation called "selection diagrams" for expressing knowledge about differences and commonalities between populations of interest and, using this representation, we reduce questions of transportability to symbolic derivations in the do-calculus. This reduction yields graph-based procedures for deciding, prior to observing any data, whether causal effects in the target population can be inferred from experimental findings in the study population. When the answer is affirmative, the procedures identify what experimental and observational findings need be obtained from the two populations, and how they can be combined to ensure bias-free transport.

Subfields: Artificial Intelligence, History and Philosophy of Science, Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In History and Philosophy of Science

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

An Analysis of the Adaptation Speed of Causal Models (2020, background): “Distribution shifts are a common problem in machine learning, as well as in causal statistics (Zhang et al., 2013; Pearl and Bareinboim, 2014).”
Toward Causal Representation Learning (2021, background): “…[115], [152], [153], [156], [167], [195], [204], [243], [263], [267], [277], [278], [281], complementing the studies of Bareinboim and Pearl [14], [185], and it inspired a thread of work in the statistics community exploiting invariance for causal discovery and other tasks [105], [106], [114],…”
Towards Causal Representation Learning (2021, background): “It also led to a range of follow-up work [279, 266, 280, 77, 114, 281, 32, 96, 263, 243, 195, 152, 156, 153, 167, 204, 115], complementing the studies of Bareinboim and Pearl [12, 185], and it inspired a thread of work in the statistics community exploiting invariance for causal discovery and other ”
Simpson's Paradox in COVID-19 Case Fatality Rates: A Mediation Analysis of Age-Related Causal Effects (2020, background): “The question whether a machine learning model will generalize outside its training distribution is closely linked to some of the concepts from causality discussed in this article and has been studied in the causal inference literature under the term “transportability” [55], [56].”
Causality for Machine Learning (2019, neutral): “al., 2018a; Lipton et al., 2018; Li et al., 2018b; Magliacane et al., 2018; Rojas-Carulla et al., 2018), and nicely complementary to studies of Elias Bareinboim and Judea (Bareinboim and Pearl, 2014; Pearl and Bareinboim, 2015). When Jonas moved to Zürich to complete and defend his Ph.D. in Statisti”
Bayesian Workflow (2020, background): “, 1972, Gelman and Hill, 2007, Finkel and Manning, 2009, Daumé, 2009), regularization to facilitate the use of large nonparametric models (Hill, 2011), and multilevel modeling for latent variables (Skrondal and Rabe-Hesketh, 2004), and there are connections between transportability and Bayesian grap”
Conformal inference of counterfactuals and individual treatment effects (2020, background): “2 Invariant prediction framework Invariant prediction is another framework proposed by Peters et al. [2016]. It is particularly powerful when there are multiple data sources under different interventions, such as in gene knockout experiments.”
Generalizing causal inferences from randomized trials: counterfactual and graphical identification (2019, methodology): “Bareinboim and Pearl have proposed general methods to assess identifiability and, when possible, to identify the post-intervention distribution in a target population by “transporting” information from other sources [47–49].”
A Review of Generalizability and Transportability (2021, methodology): “A corresponding set of assumptions under the structural equation model (SEM) framework has also been derived (Pearl and Bareinboim, 2014; Pearl, 2015; Pearl and Bareinboim, 2011; Bareinboim and Pearl, 2014; Bareinboim and Tian, 2015; Bareinboim and Pearl, 2016; Correa, Tian and Bareinboim, 2018).”
Biases in Generative Art: A Causal Look from the Lens of Art History (2020, background): “can be transferred into to a new population in which only observational studies can be conducted [Pearl and Bareinboim, 2014].”
Using Experiments to Correct for Selection in Observational Studies (2020, neutral): “We have two samples to draw on to estimate τ P , as in the literature on combining datasets, e.g., Hotz et al. [2005], Ridder and Moffitt [2007], Pearl et al. [2014].”