Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond

By Amir Feder, Katherine A. Keith, Emaad A. Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood-Doughty, Jacob Eisenstein, Justin Grimmer, Roi Reichart, Margaret E. Roberts and 3 more. Transactions of the Association for Computational Linguistics, 2021

330 citations recorded by the source corpus.

Abstract A fundamental goal of scientific research is to learn about causal relationships. However, despite its critical role in the life and social sciences, causality has not had the same importance in Natural Language Processing (NLP), which has traditionally placed more emphasis on predictive tasks. This distinction is beginning to fade, with an emerging area of interdisciplinary research at the convergence of causal inference and language processing. Still, research on causality in NLP remains scattered across domains without unified definitions, benchmark datasets and clear articulations of the challenges and opportunities in the application of causal inference to the textual domain, with its unique properties. In this survey, we consolidate research across academic areas and situate it in the broader NLP landscape. We introduce the statistical challenge of estimating causal effects with text, encompassing settings where text is used as an outcome, treatment, or to address confounding. In addition, we explore potential uses of causal inference to improve the robustness, fairness, and interpretability of NLP models. We thus provide a unified overview of causal inference for the NLP community.1

Subfields: Artificial Intelligence, General Social Sciences

View Paper: https://www.semanticscholar.org/paper/130d432ccbc836380a212bea618f84ff094a6a52

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Social Sciences

Citations in the corpus

Measuring Causal Effects of Data Statistics on Language Model's 'Factual' Predictions (2022, background): “We refer to Feder et al. (2021a) for an overview of causal-based approaches and challenges in NLP.”
Informativeness and Invariance: Two Perspectives on Spurious Correlations in Natural Language (2022, methodology): “…data generating process, we can compute an interventional distribution P ( Y | do ( X 1 := x ) , X 2 , X 3 ) , which corresponds to the distribution over Y in a data generating process in which the variable X 1 is surgically set to the value x (Pearl, 1995; Peters et al., 2017; Feder et al., 2021).”