A Right to Reasonable Inferences: Re-Thinking Data Protection Law in the Age of Big Data and AI

By Sandra Wachter, Brent Daniel Mittelstadt. 2018

434 citations recorded by the source corpus.

Big Data analytics and artificial intelligence (AI) draw non-intuitive and unverifiable inferences and predictions about the behaviors, preferences, and private lives of individuals. These inferences draw on highly diverse and feature-rich data of unpredictable value, and create new opportunities for discriminatory, biased, and invasive decision-making. Concerns about algorithmic accountability are often actually concerns about the way in which these technologies draw privacy invasive and non-verifiable inferences about us that we cannot predict, understand, or refute.Data protection law is meant to protect people’s privacy, identity, reputation, and autonomy, but is currently failing to protect data subjects from the novel risks of inferential analytics. The broad concept of personal data in Europe could be interpreted to include inferences, predictions, and assumptions that refer to or impact on an individual. If seen as personal data, individuals are granted numerous rights under data protection law. However, the legal status of inferences is heavily disputed in legal scholarship, and marked by inconsistencies and contradictions within and between the views of the Article 29 Working Party and the European Court of Justice.As we show in this paper, individuals are granted little control and oversight over how their personal data is used to draw inferences about them. Compared to other types of personal data, inferences are effectively ‘economy class’ personal data in the General Data Protection Regulation (GDPR). Data subjects’ rights to know about (Art 13-15), rectify (Art 16), delete (Art 17), object to (Art 21), or port (Art 20) personal data are significantly curtailed when it comes to inferences, often requiring a greater balance with controller’s interests (e.g. trade secrets, intellectual property) than would otherwise be the case. Similarly, the GDPR provides insufficient protection against sensitive inferences (Art 9) or remedies to challenge inferences or important decisions based on them (Art 22(3)).This situation is not accidental. In standing jurisprudence the European Court of Justice (ECJ; Bavarian Lager, YS. and M. and S., and Nowak) and the Advocate General (AG; YS. and M. and S. and Nowak) have consistently restricted the remit of data protection law to assessing the legitimacy of input personal data undergoing processing, and to rectify, block, or erase it. Critically, the ECJ has likewise made clear that data protection law is not intended to ensure the accuracy of decisions and decision-making processes involving personal data, or to make these processes fully transparent.Conflict looms on the horizon in Europe that will further weaken the protection afforded to data subjects against inferences. Current policy proposals addressing privacy protection (the ePrivacy Regulation and the EU Digital Content Directive) fail to close the GDPR’s accountability gaps concerning inferences. At the same time, the GDPR and Europe’s new Copyright Directive aim to facilitate data mining, knowledge discovery, and Big Data analytics by limiting data subjects’ rights over personal data. And lastly, the new Trades Secrets Directive provides extensive protection of commercial interests attached to the outputs of these processes (e.g. models, algorithms and inferences).In this paper we argue that a new data protection right, the ‘right to reasonable inferences’, is needed to help close the accountability gap currently posed ‘high risk inferences’ , meaning inferences that are privacy invasive or reputation damaging and have low verifiability in the sense of being predictive or opinion-based. In cases where algorithms draw ‘high risk inferences’ about individuals, this right would require ex-ante justification to be given by the data controller to establish whether an inference is reasonable. This disclosure would address (1) why certain data is a relevant basis to draw inferences; (2) why these inferences are relevant for the chosen processing purpose or type of automated decision; and (3) whether the data and methods used to draw the inferences are accurate and statistically reliable. The ex-ante justification is bolstered by an additional ex-post mechanism enabling unreasonable inferences to be challenged. A right to reasonable inferences must, however, be reconciled with EU jurisprudence and counterbalanced with IP and trade secrets law as well as freedom of expression and Article 16 of the EU Charter of Fundamental Rights: the freedom to conduct a business.

Subfields: Artificial Intelligence, Law, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Law

In Sociology and Political Science

Citations in the corpus

Actionable Recourse in Linear Classification (2018, background): “ual rights with respect to algorithmic decision-making are often motivated by the need for agency over machine-made decisions (e.g., autonomy is a core motivation for dataprotectionasdiscussedby,e.g.,[37]).Recoursereflectsaprecise notion of agency, namely the ability to meaningfully influence a deci”
Towards Transparency by Design for Artificial Intelligence (2020, methodology): “…individuals will have against problematic ADM. Scholars have highlighted that discriminatory practices and potential chilling effects arising from the employment of such technologies are inadequately covered in the law, like data protection law (Wachter and Mittelstadt 2019; Büchi et al. 2019).”
Transparency in artificial intelligence (2020, background): “Hence, Wachter & Mittelstadt (2018) argue for a “right to reasonable inferences”.”
This Thing Called Fairness (2019, background): “Research by social scientists and humanities scholars has similarly advanced the state of theory and application [5, 11, 80, 136, 149].”
Asynchronous Federated Learning on Heterogeneous Devices: A Survey (2021, background): “Notably, the emphasis on preserving privacy during data sharing continues to persist, with newly enacted legislation and regulations further complicating the process of data acquisition [4, 5, 6].”
The Ethical Implications of Using Artificial Intelligence in Auditing (2020, background): “Wachter and Mittelstadt (2019) observe that “data protection law is meant to protect people’s privacy, identity, reputation, and autonomy, but is currently failing to protect data subjects from the novel risks of inferential analytics.””
On the apparent conflict between individual and group fairness (2019, background): “They may miss unfairness against people as a result of their being at the intersection of multiple kinds of discrimination [15], or groups which are not (yet) defined in anti-discrimination law but may need protecting [49].”
Repairing without Retraining: Avoiding Disparate Impact with Counterfactual Distributions (2019, background): “It also requires training a model to detect membership in the target class, which can be problematic if the model is deployed for other purposes (see e.g., Wachter and Mittelstadt, 2018, for a discussion).”
Transparency you can trust: Transparency requirements for artificial intelligence between legal norms and contextual concerns (2019, methodology): “Given the current debate about the right to reasonable inferences in the context of the GDPR (Wachter and Mittelstadt, 2019), the legal analysis focuses strongly on the European context.”
Principles alone cannot guarantee ethical AI (2019, neutral): “However, the impact of such frameworks is variable and limited in scope 26,27 .”
"Happy and Assured that life will be easy 10years from now.": Perceptions of Artificial Intelligence in 8 Countries (2019, background): “repurposing [57], and privacy-sensitive inferences in systems that do not support explanation or refutation [60].”