Using Text Embeddings for Causal Inference

By Victor Veitch, Dhanya Sridhar, David M. Blei. 2019

20 citations recorded by the source corpus.

We address causal inference with text documents. For example, does adding a theorem to a paper affect its chance of acceptance? Does reporting the gender of a forum post author affect the popularity of the post? We estimate these effects from observational data, where they may be confounded by features of the text such as the subject or writing quality. Although the text suffices for causal adjustment, it is prohibitively high-dimensional. The challenge is to find a low-dimensional text representation that can be used in causal inference. A key insight is that causal adjustment requires only the aspects of text that are predictive of both the treatment and outcome. Our proposed method adapts deep language models to learn low-dimensional embeddings from text that predict these values well; these embeddings suffice for causal adjustment. We establish theoretical properties of this method. We study it empirically on semi-simulated and real data on paper acceptance and forum post popularity. Code is available at this https URL.

Subfields: Artificial Intelligence, Management Science and Operations Research, Natural Language Processing, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Natural Language Processing

In Causal Inference & Discovery

Citations in the corpus

Unsupervised Discovery of Implicit Gender Bias (2020, background): “In the first scenario, text maybe be a confounding variable that needs to be controlled in order to measure the effect of a non-text variable (Roberts et al., Forthcoming; Veitch et al., 2019).”
Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates (2020, methodology): “To construct Table 1, we started with three seed papers: Roberts et al. (2020), Veitch et al. (2019), and Wood-Doughty et al. (2018).”