Iterative Orthogonal Feature Projection for Diagnosing Bias in Black-Box Models

By Julius Adebayo, Lalana Kagal. 2016

68 citations recorded by the source corpus.

Predictive models are increasingly deployed for the purpose of determining access to services such as credit, insurance, and employment. Despite potential gains in productivity and efficiency, several potential problems have yet to be addressed, particularly the potential for unintentional discrimination. We present an iterative procedure, based on orthogonal projection of input attributes, for enabling interpretability of black-box predictive models. Through our iterative procedure, one can quantify the relative dependence of a black-box model on its input attributes.The relative significance of the inputs to a predictive model can then be used to assess the fairness (or discriminatory extent) of such a model.

Subfields: Artificial Intelligence, Economics and Econometrics, Safety Research, Safety, Robustness & Interpretability, AI Ethics, Fairness & Society

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Economics and Econometrics

In Safety Research

In Safety, Robustness & Interpretability

In AI Ethics, Fairness & Society

Citations in the corpus

A Survey of Methods for Explaining Black Box Models (2018, methodology): “2016 Inspection PDP AGN TAB X X X X OPIA [1] Adebayo et al.”
Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation (2017, methodology): “ February 2–3, 2018, New Orleans, LA, USA © 2018 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-6012-8/18/02. https://doi.org/10.1145/3278721.3278725 Several approaches have been proposed [1, 2, 13, 17, 20, 33] to audit black-box risk scoring models: remove, permute, or obscure a protect”
Empirical Risk Minimization under Fairness Constraints (2018, methodology): “nce of relaxation exploiting spectral decompositions. Finally, the third family of methods implements fairness by modifying the data representation and then employs standard machine learning methods: [31, 7, 32, 4, 33, 34] are examples of these methods but, again, no consistency property nor compari”
Fairness in Machine Learning (2020, unknown): “MTL jointly learns a shared model 𝑤 0 and task speciﬁc models 𝑤 𝑠 = 𝑤 0 + 𝑣 𝑠 ∀ 𝑠 ∈ S by encouraging them to be close to each other.”
Principles and Practice of Explainable Machine Learning (2020, background): “Additional ways to assess the importance of a feature can also be found, such as the one in [1].”
Fairness testing: testing software for discrimination (2017, background): “Restricting the input space to real-world inputs [1] may similarly hide software discrimination that causal testing can reveal.”
On the Direction of Discrimination: An Information-Theoretic Analysis of Disparate Impact in Machine Learning (2018, background): “These issues have motivated a growing stream of technical work on disparate impact in ML, focusing on topics such as: (i) how to identify and quantify disparate impact [8, 9, 10, 11]; (ii) how to train models that mitigate disparate impact [12, 13, 14]; and (iii) how to identify causal factors of di”
Auditing Black-Box Models Using Transparent Model Distillation With Side Information (2017, background): “Adebayo and Kagal [1] also learn their own risk scoring models when the black-box model cannot be queried.”
Explainable Deep Learning: A Field Guide for the Uninitiated (2020, methodology): “Pre-processing methods (Kamiran & Calders, 2010, 2012; Zemel et al., 2013; Louizos et al., 2015; Adebayo & Kagal, 2016; Calmon et al., 2017; Gordaliza et al., 2019) learn an alternative representation of the input data that removes information correlated to the sensitive attributes (such as race or…”
Themis: automatically testing software for discrimination (2018, background): “FairML [1] uses orthogonal projection to co-perturb attributes, which can mask some discrimination, but find discrimination that is more likely to be observed in real-world scenarios.”