From Dependence to Causation

By David Lopez-Paz. 2016

30 citations recorded by the source corpus.

Machine learning is the science of discovering statistical dependencies in data, and the use of those dependencies to perform predictions. During the last decade, machine learning has made spectacular progress, surpassing human performance in complex tasks such as object recognition, car driving, and computer gaming. However, the central role of prediction in machine learning avoids progress towards general-purpose artificial intelligence. As one way forward, we argue that causal inference is a fundamental component of human intelligence, yet ignored by learning algorithms. Causal inference is the problem of uncovering the cause-effect relationships between the variables of a data generating system. Causal structures provide understanding about how these systems behave under changing, unseen environments. In turn, knowledge about these causal dynamics allows to answer "what if" questions, describing the potential responses of the system under hypothetical manipulations and interventions. Thus, understanding cause and effect is one step from machine learning towards machine reasoning and machine intelligence. But, currently available causal inference algorithms operate in specific regimes, and rely on assumptions that are difficult to verify in practice. This thesis advances the art of causal inference in three different ways. First, we develop a framework for the study of statistical dependence based on copulas and random features. Second, we build on this framework to interpret the problem of causal inference as the task of distribution classification, yielding a family of novel causal inference algorithms. Third, we discover causal structures in convolutional neural network features using our algorithms. The algorithms presented in this thesis are scalable, exhibit strong theoretical guarantees, and achieve state-of-the-art performance in a variety of real-world benchmarks.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Invariant Risk Minimization (2019, background): “Understanding which patterns are useful has been previously studied as a correlation-versus-causation dilemma, since spurious correlations stemming from data biases are unrelated to the causal explanation of interest [31, 27, 35, 52].”
Causal Generative Neural Networks (2017, methodology): “Using the linear MMD approximation [Lopez-Paz, 2016], CGNN ( (cid:92) MMD m k as explained in Section 3.2) reduces the cost by a factor of 5 without hindering the performance.”
When Worlds Collide: Integrating Different Counterfactual Assumptions in Fairness (2017, background): “Questions requiring the joint distribution of counterfactuals are hard to answer, as they demand partially untestable “cross-world” assumptions [5, 17], and even many of the empirically testable assumptions cannot be falsified from observational data alone [14], requiring possibly infeasible randomi”
Large-scale kernel methods for independence testing (2016, methodology): “More speciﬁcally, they develop Randomised Canonical Correlation Analysis (RCCA) (see also [23, 21]) approximating the non-linear kernel-based generalisation of the Canonical Correlation Analysis [20, 4] and using a further copula transformation, construct a test statistic termed RDC (randomised…”
Single-Model Uncertainties for Deep Learning (2018, background): “Third, uncertainty estimation is important when analyzing noise structure, such as in causal discovery [52] and in the estimation of predictive intervals.”
Causal Regularization (2017, background): “Recently several algorithms have been proposed that distinguish between the cause and effect based on the natural assumption that steps in the process that generates the data are independent from each other, see (Lemeire and Dirkx, 2006; Janzing et al., 2012; Daniusis et al., 2010; Lopez-Paz, 2016; ”
Explainable and Interpretable Models in Computer Vision and Machine Learning (2017, unknown): “Using the linear MMD approximation Lopez-Paz (2016), CGNN ( (cid:92) MMD m k as explained in appendix 8.1) reduces the cost by a factor of 5 without hindering the performance.”