Transparent to whom? No algorithmic accountability without a critical audience

By Jakko Kemper, Daan Kolkman. 2018

195 citations recorded by the source corpus.

ABSTRACT Big data and data science transform organizational decision-making. We increasingly defer decisions to algorithms because machines have earned a reputation of outperforming us. As algorithms become embedded within organizations, they become more influential and increasingly opaque. Those who create algorithms may make arbitrary decisions in all stages of the ‘data value chain’, yet these subjectivities are obscured from view. Algorithms come to reflect the biases of their creators, can reinforce established ways of thinking, and may favour some political orientations over others. This is a cause for concern and calls for more transparency in the development, implementation, and use of algorithms in public- and private-sector organizations. We argue that one elementary – yet key – question remains largely undiscussed. If transparency is a primary concern, then to whom should algorithms be transparent? We consider algorithms as socio-technical assemblages and conclude that without a critical audience, algorithms cannot be held accountable.

Subfields: Information Systems, Management Information Systems, Safety Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

In Management Information Systems

In Safety Research

Citations in the corpus

How Computers See Gender (2019, unknown): “…on gender classification has highlighted numerous methods for identifying gender (e.g. periocular regions [68]; facial morphology [90]; lips, eyes, and cheeks [76]), but how, when, and by whom gender is embedded into the pipeline of data, labels, and models is opaque to outsiders (e.g. [56, 89]).”
Transparency you can trust: Transparency requirements for artificial intelligence between legal norms and contextual concerns (2019, methodology): “As work in critical algorithm studies has pointed out, transparency practices do not take place in a social vacuum but play particular roles in their speciﬁc cultural and organizational settings (Beer, 2017; Kemper and Kolkman, 2018).”
How Platform-User Power Relations Shape Algorithmic Accountability: A Case Study of Instant Loan Platforms and Financially Stressed Users in India (2022, background): “and the responsiveness of technology providers [17, 60, 76].”
Reasons, Values, Stakeholders: A Philosophical Framework for Explainable Artificial Intelligence (2021, background): “1 More concretely, the development of explainable AI systems is motivated by several social and ethical reasons (see, for example, [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]): increasing societal acceptance of prediction-based decisions, establishing trust in the results of these decisions, making…”
User Perceptions of Algorithmic Decisions in the Personalized AI System:Perceptual Evaluation of Fairness, Accountability, Transparency, and Explainability (2020, background): “Levels of transparency, fairness, and accuracy are perceived as a function of user disposition (Kemper & Kolkman, 2019).”
How Data Science Workers Work with Data: Discovery, Capture, Curation, Design, Creation (2019, background): “We take a similar interest to make visible how data work in contemporary data science [51], [74], [79].”
What to account for when accounting for algorithms: a systematic literature review on algorithmic accountability (2020, background): “Kemper and Kolkman [86] argue that one cannot give account without the audience understanding the subject matter and being able to engage with the material in a critical way.”