Algorithmic Transparency via Quantitative Input Influence: Theory and Experiments with Learning Systems

By Anupam Datta, Shayak Sen, Yair Zick. 2016

783 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Safety Research, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Safety Research

In Safety, Robustness & Interpretability

Citations in the corpus

Axiomatic Attribution for Deep Networks (2017, methodology): “by Shapley-Shubik (Shapley & Shubik, 1971) from the cost sharing literature, and used by (Lundberg & Lee, 2016; Datta et al., 2016) to compute feature attributions (though they were not studying deep networks).”
Understanding Black-box Predictions via Influence Functions (2017, background): “, 2016) or by perturbing the test point to see how the prediction changes (Simonyan et al., 2013; Li et al., 2016b; Datta et al., 2016; Adler et al., 2016).”
Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “The work in [173] presents a broad variety of measures to tackle the quantiﬁcation of the degree Fig.”
Mitigating Sybils in Federated Learning Poisoning (2018, background): “Recent work on feature influence in deep neural networks [1], [15] may better capture the intent of sybil-based poisoning attacks in deep learning and we leave this analysis as future work.”
Visualizing the Feature Importance for Black Box Models (2018, background): “Similar work proposing a general notion of a quantity of interest for the characteristic function of the Shapley value and focusing on the joint and marginal contributions of feature sets was introduced by [8].”
Consistent Individualized Feature Attribution for Tree Ensembles (2018, methodology): “SHAP values are theoretically optimal, but like other model agnostic feature attribution methods [2, 9, 21, 27], they can be challenging to compute.”
Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation (2017, background): “ February 2–3, 2018, New Orleans, LA, USA © 2018 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-6012-8/18/02. https://doi.org/10.1145/3278721.3278725 Several approaches have been proposed [1, 2, 13, 17, 20, 33] to audit black-box risk scoring models: remove, permute, or obscure a protect”
A Unified Approach to Interpreting Model Predictions (2017, methodology): “Classic Shapley value estimation: There are three previous methods that use classic equations from cooperative game theory to compute explanations of model predictions: Shapley regression values [3], Shapley sampling values [8], and Quantitative Input Influence [2].”
Auditing black-box models for indirect influence (2016, background): “Datta et al. [8] provide a generalization of this idea, linking it to game-theoretic notions of inﬂuence and showing that different choices of probability spaces and random variables yield a number of different interesting auditing measures.”
Do Not Trust Additive Explanations (2019, background): “Another coalition-based measure of variable influence is Quantitative Input Influence (QII) (Datta et al., 2016).”
All Models are Wrong, but Many are Useful: Learning a Variable’s Importance by Studying an Entire Class of Prediction Models Simultaneously (2018, background): “…are provided at http://jmlr.org/papers/v20/18-760.html. the decrease in prediction accuracy when a covariate is permuted (Breiman, 2001; Breiman et al., 2001; see also Strobl et al., 2008; Altmann et al., 2010; Zhu et al., 2015; Gregorutti et al., 2015; Datta et al., 2016; Gregorutti et al., 2017).”
Interpreting Blackbox Models via Model Extraction (2017, methodology): “There has been work on constructing global explanations: relative influence scores the contribution of each feature in random forests (Friedman, 2001), and (Datta et al., 2016) uses the Shapley value to quantify feature influence.”
Explaining Explanations in AI (2018, methodology): “…seek to explain how a fixed model leads to a particular prediction, either by fitting a simpler, local model around a particular decision (Ribeiro et al., 2016), or by perturbing variables to measure how the prediction changes (Adler et al., 2016; Datta et al., 2016; Simonyan et al., 2013).”
Learning Explainable Models Using Attribution Priors (2019, background): “Previous literature about such methods has focused on the axioms they should satisfy (Lundberg and Lee, 2017; Sundararajan et al., 2017; Štrumbelj and Kononenko, 2014; Datta et al., 2016), and how attribution methods can give us insight into model behavior (Lundberg et al.”
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions (2018, methodology): “Input Influence : Presents a list of input variables alongside a quantitative measure of their ‘influence’, positive or negative, on a decision (e.g. [13, 86]) 2.”
Data Shapley: Equitable Valuation of Data for Machine Learning (2019, background): “Shapley value has been recently proposed as a feature importance score for the purpose of interpreting black-box predictive models (Kononenko et al., 2010; Datta et al., 2016; Lundberg & Lee, 2017; Cohen et al., 2007; Chen et al., 2018; Lundberg et al., 2018).”