Structural Intervention Distance (SID) for Evaluating Causal Graphs

By Jan Peters, Peter Buhlmann. 2013

49 citations recorded by the source corpus.

Causal inference relies on the structure of a graph, often a directed acyclic graph (DAG). Different graphs may result in different causal inference statements and different intervention distributions. To quantify such differences, we propose a (pre-) distance between DAGs, the structural intervention distance (SID). The SID is based on a graphical criterion only and quantifies the closeness between two DAGs in terms of their corresponding causal inference statements. It is therefore well-suited for evaluating graphs that are used for computing interventions. Instead of DAGs it is also possible to compare CPDAGs, completed partially directed acyclic graphs that represent Markov equivalence classes. Since it differs significantly from the popular Structural Hamming Distance (SHD), the SID constitutes a valuable additional measure. We discuss properties of this distance and provide an efficient implementation with software code available on the first author's homepage (an R package is under construction).

Subfields: Artificial Intelligence, Statistics and Probability, Causal Inference & Discovery

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

In Causal Inference & Discovery

Citations in the corpus

Differentiable Causal Discovery from Interventional Data (2020, methodology): “The performance of each method is assessed by two metrics comparing the estimated graph to the ground truth graph: i) the structural Hamming distance (SHD) which is simply the number of edges that differ between two DAGs (either reversed, missing or superfluous) and ii) the structural interventional”
Gradient-Based Neural DAG Learning (2019, methodology): “For each approach, we evaluate the estimated graph on two metrics: the structural hamming distance (SHD) and the structural interventional distance (SID) [25].”
Causal Generative Neural Networks (2017, background): “…(Area Under the Precision/Recall curve), the Structural Hamming Distance (SHD, the number of edge modiﬁcations to transform one graph into another) and the Structural Intervention Distance (SID, the number of equivalent two-variable interventions between two graphs) [Peters and B¨uhlmann, 2013].”
Kernel‐based tests for joint independence (2016, methodology): “For example, the causal hypothesis ‘X is a causal ancestor of Y’ can in principle be tested by applying the DAG verification method to all DAGs satisfying this ancestor relationship.”
CAM: Causal Additive Models, high-dimensional order search and penalized regression (2013, background): “As a performance measure, we consider the recently proposed structural intervention distance (SID); see [24].”
A survey of Bayesian Network structure learning (2021, background): “Other structural metrics have focused on assessing the learnt graph in terms of causal effects, such as the Structural Intervention Distance (SID) by Peters and Buhlmann (2015) which measures the closeness of two DAGs in terms of their corresponding causal inference statements.”
UvA-DARE ( Digital Academic Repository ) Causal Discovery with Continuous Additive Noise Models (2014, unknown): “We also report analogous results for the structural intervention distance (SID), which has recently been proposed (Peters and Bühlmann, 2013).”
Partition MCMC for Inference on Acyclic Digraphs (2015, background): “In the context of causal graphs, a distance was recently defined (Peters and Bühlmann 2015), though purely based on a graphical criterion, directly related to the network interpretation in terms of differences in the causal inference statements deriving from each structure.”
D’ya Like DAGs? A Survey on Structure Learning and Causal Discovery (2021, methodology): “Structural Interventional Distance (SID) [187]: Is a count of the number of vertex pairs (i, j) for which the intervention p (x j |do (Xi = x)) would be incorrect if the estimated graph (as opposed to the ground-truth graph) were used for the associated adjustment set.”
Explainable and Interpretable Models in Computer Vision and Machine Learning (2017, unknown): “As shown on Table 4 (right) in Appendix, and as could be expected, the scores of all algorithms ) (Peters and B¨uhlmann, 2013) is shown on Table 4 in section in section ”Table of Scores for the Experiments on Graphs” in Appendix. are lower when spurious edges are introduced.”
A Complete Generalized Adjustment Criterion (2015, background): “In turn, these results could then be used to characterize distances between graphs, as done by Peters and Bühlmann (2015).”