Discovery of Treatments from Text Corpora

By Christian Fong, Justin Grimmer. 2016

54 citations recorded by the source corpus.

An extensive literature in computational social science examines how features of messages, advertisements, and other corpora affect individuals’ decisions, but these analyses must specify the relevant features of the text before the experiment. Automated text analysis methods are able to discover features of text, but these methods cannot be used to obtain the estimates of causal effects—the quantity of interest for applied researchers. We introduce a new experimental design and statistical model to simultaneously discover treatments in a corpora and estimate causal effects for these discovered treatments. We prove the conditions to identify the treatment effects of texts and introduce the supervised Indian Buffet process to discover those treatments. Our method enables us to discover treatments in a training set using a collection of texts and individuals’ responses to those texts, and then estimate the effects of these interventions in a test set of new texts and survey respondents. We apply the model to an experiment about candidate biographies, recovering intuitive features of voters’ decisions and revealing a penalty for lawyers and a bonus for military service.

Subfields: General Social Sciences, Political Science and International Relations, Sociology and Political Science, Natural Language Processing, Medical & Health AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In General Social Sciences

In Political Science and International Relations

In Sociology and Political Science

In Natural Language Processing

In Medical & Health AI

Roi Reichart

Citations in the corpus

CausaLM: Causal Model Explanation Through Counterfactual Language Models (2020, neutral): “ often binary (D’Amour et al. 2017; Athey et al. 2017). Recently, this seems to be changing, with substantial work being done on the intersection of causal inference and NLP (Tan, Lee, and Pang 2014; Fong and Grimmer 2016; Egami et al. 2018; Wood-Doughty, Shpitser, and Dredze 2018; Veitch, Sridhar, ”
Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates (2020, background): “, 2019), text as treatment (Fong and Grimmer, 2016; Egami et al.; Wood-Doughty et al., 2018; Tan et al., 2014), text as outcome (Egami et al.”
How to make causal inferences using texts (2018, unknown): “To select a final model, we then evaluate the candidate model fits using a model fit statistic introduced in Fong and Grimmer ( 17 ).”
Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond (2021, background): “Such concerns could be avoided by randomly assigning texts to readers (Fong and Grimmer, 2016, 2021), but that may be impractical.”