European Union Regulations on Algorithmic Decision-Making and a "Right to Explanation"
By Bryce Goodman, Seth R. Flaxman. 2016
2,239 citations recorded by the source corpus.
We summarize the potential impact that the European Union’s new General Data Protection Regulation will have on the routine use of machine learning algorithms. Slated to take effect as law across the EU in 2018, it will restrict automated individual decision-making (that is, algorithms that make decisions based on user-level predictors) which “significantly affect” users. The law will also effectively create a “right to explanation,” whereby a user can ask for an explanation of an algorithmic decision that was made about them. We argue that while this law will pose large challenges for industry, it highlights opportunities for computer scientists to take the lead in designing algorithms and evaluation frameworks which avoid discrimination and enable explanation.
Subfields: Artificial Intelligence, Health Informatics, Safety Research, Safety, Robustness & Interpretability, AI Ethics, Fairness & Society
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Percy Liang
- Mohit Bansal
- Yoshua Bengio
- Quoc V. Le
- Jianfeng Gao
- Luke S. Zettlemoyer
- Yonatan Belinkov
- Graham Neubig
In Health Informatics
- Timo Speith
- Emilio Ferrara
- Qiang Yang
- Luciano Floridi
- Joao Marques-Silva
- Alexey Ignatiev
- James Y. Zou
- Jessica Morley
In Safety Research
- Emilio Ferrara
- Vincent C. Müller
- Timo Speith
- Jessica Morley
- Anat Elhalal
- Libby Kinsey
- Nicolas Papernot
- Patrick Drew McDaniel
In Safety, Robustness & Interpretability
- Been Kim
- Wojciech Samek
- Nicolas Papernot
- K. Müller
- Nicholas Carlini
- Grgoire Montavon
- Longo Luca
- Giulia Vilone
In AI Ethics, Fairness & Society
- Cynthia Rudin
- R. Caruana
- Sam Corbett-Davies
- Joao Marques-Silva
- Alexey Ignatiev
- Nina Narodytska
- Eva Thelisson
- Marcel A. J. van Gerven
Citations in the corpus
- The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation (2018, background): “As an example of policymaking in this domain that has surfaced several difficulties, the European Union’s General Data Protection Regulation is a commonly-discussed example of a policy that is hard to interpret and apply in the context of current machine learning algorithms (Goodman and Flaxman, 201”
- Towards A Rigorous Science of Interpretable Machine Learning (2017, background): “…criteria such as safety [Otte, 2013, Amodei et al., 2016, Varshney and Alemzadeh, 2016], nondiscrimination [Bostrom and Yudkowsky, 2014, Ruggieri et al., 2010, Hardt et al., 2016], avoiding technical debt [Sculley et al., 2015], or providing the right to explanation [Goodman and Flaxman, 2016].”
- Understanding Black-box Predictions via Influence Functions (2017, background): “By understanding why a model does what it does, we can hope to improve the model (Amershi et al., 2015), discover new science (Shrikumar et al., 2016), and provide end-users with explanations of actions that impact them (Goodman & Flaxman, 2016).”
- Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, positive): “In a certain literature strand, an explainable ML model suggests a clear visualization of the relations affecting a result, allowing for a fairness or ethical analysis of the model at hand [3,100] .”
- Explaining Explanations: An Overview of Interpretability of Machine Learning (2018, background): “With impending regulations like the European Union’s “Right to Explanation” [12], calls for diversity and inclusion in AI systems [13], findings that some automated systems may reinforce inequality and bias [14], and requirements for safe and secure AI in safety-critical tasks [15], there has been a…”
- SecureBoost: A Lossless Federated Learning Framework (2019, background): “GDPR is designed to give users more control over their personal data [1], [2], [3], [4].”
- The mythos of model interpretability (2016, background): “Few would suggest that a human could contemplate a thousand parameters at once, but most could simulate a linear model with 20 parameters.”
- Fairwashing: the risk of rationalization (2019, background): “Since the right to explanation as defined in current regulations [15] does not give precise directives on what it means to provide a “valid explanation” [33, 13], there is a legal loophole that can be used by dishonest companies to cover up the possible unfairness of their black-box models by…”
- Metrics for Explainable AI: Challenges and Prospects (2018, background): “European Union regulations on algorithmic decisionmaking and a “right to explanation.””
- Sanity Checks for Saliency Maps (2018, background): “Explanations could potentially help satisfy regulatory requirements [3], help practitioners debug their model [4, 5], and perhaps, reveal bias or other unintended effects learned by a model [6, 7].”
- Learning Optimal Decision Trees with SAT (2018, background): “Furthermore, the relevance of XAI is implicit in recent EU-level regulations [EU Data Protection Regulation, 2016], which is motivated in part by a demand for greater transparency in algorithmic decision making [Goodman and Flaxman, 2017; Weller, 2017].”
- Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning (2018, background): “Nevertheless, legislation like the European Union’s General Data Protection Regulation require that companies deploying ML and other forms of analysis on certain sensitive data provide such interpretable outputs if the prediction made by a ML model is used to make decisions without a human in the lo”
- Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) (2017, background): “With increasing demands for more explainable ML (Goodman & Flaxman, 2016), there is an growing need for methods that can be applied without retraining or modifying the network.”
- Interpreting Classifiers through Attribute Interactions in Datasets (2017, background): “Gaining insight into machine learning models is a topic that will become more important in the future, e.g., due to possible legislative requirements (Goodman & Flaxman, 2016).”
- On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products (2016, background): “Moreover, not only is non-harm the first ethical duty, many of the safety issues for machine learning we have discussed in this paper are starting to enter legal obligati ons as well [45].”
- How algorithmic confounding in recommendation systems increases homogeneity and decreases utility (2017, background): “Due to these and other concerns, regulations are emerging to restrict automated individual decision-making, such as recommendation systems [21]; this work will aid in making such eorts eective.”