Algorithmic Accountability Reporting: On the Investigation of Black Boxes

By Nicholas A. Diakopoulos. 2014

242 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

Citations in the corpus

How the machine ‘thinks’: Understanding opacity in machine learning algorithms (2016, background): “In some ways this extends the approach of the external audit proposed by Sandvig et al. (2014) and by Diakopoulos (2013) using sophisticated algorithmic implementations.”
Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing (2020, methodology): “Building from methods of external auditing in investigative journalism and research [17, 62, 65], algorithmic auditing has started to become similar in spirit to the well-established practice of bug bounties, where external hackers are paid for finding vulnerabilities and bugs in released software…”
Addressing Age-Related Bias in Sentiment Analysis (2018, methodology): “Researchers have described the algorithms that drive many of the systems we use as “black boxes” [Diakopoulos, 2014].”
Artificial Intelligence and Journalism (2019, background): “…citing the excellent work that has been done in the past few years by scholars looking at the field of data journal-ism and how AI is being adopted inside it (Diakopoulos, 2013, 2014; Fink & Anderson, 2014; Hamilton & Turner, 2009; Howard, 2014; Lewis & Westlund, 2015; Parasie, 2015; Royal, 2010).”
Algorithms as culture: Some tactics for the ethnography of algorithmic systems (2017, background): “These projects, inspired by historical eﬀorts to expose housing discrimination, treat algorithmic systems as black-box functions: hidden operations that turn inputs into outputs, like mortgage applications into home loans (Diakopoulos, 2013; Sandvig et al., 2014).”
Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda (2018, background): “Journalism can also play a role to hold algorithms accountable by reporting issues of bias and algorithmic power to the broader public [43].”
Human-AI Interactions in Public Sector Decision-Making:"Automation Bias"and"Selective Adherence"to Algorithmic Advice (2021, background): “…relied upon in areas as varied as policing, welfare, criminal justice, healthcare, immigration, or education (Busuioc 2021; Calo and Citron 2021; Diakopoulos 2014; Eubanks 2018; Engstrom et al. 2020; O’Neil 2016; Richardson, Schultz, and Crawford 2019; Veale and Brass 2019; Yeung and Lodge…”
Thinking critically about and researching algorithms (2014, background): “As Diakopoulos (2013, p. 2) argues: ‘ We ’ re living in a world now where algorithms adjudicate more and more consequential decisions in our lives.”
Algorithmic IF … THEN rules and the conditions and consequences of power (2017, background): “These concerns lead to various calls (Diakopoulos, 2013; Kitchin, 2014; Seaver, 2013) for algorithms to be held to account, governed and regulated.”
Auditing Black-box Models by Obscuring Features (2016, background): “While fields outside of computer science have begun paying attention to the idea of auditing blackbox machine learning algorithms [15, 5], within computer science this question has mostly been”
Fact Checking: Task definition and dataset construction (2014, background): “The main drawback of the baseline approaches mentioned (aside from their potential coverage) is the lack of interpretability of their verdicts, also referred to as algorithmic accountability (Diakopoulos, 2014).”
"I always assumed that I wasn't really that close to [her]": Reasoning about Invisible Algorithms in News Feeds (2015, background): “2702556 even their perception of the world [9].”
Identifying Platform Effects in Social Media Data (2016, background): “Outside accounting via testing inputs (Diakopoulos 2014) is an important way of identifying overall effective outcomes, but such cross-sectional audits lack a baseline to know how much a given platform design successfully shapes behavior.”
Bearing Account-able Witness to the Ethical Algorithmic System (2016, background): “In popular discussions, algorithmic accountability is framed in terms of openness and transparency, at least theoretically enabling the algorithm, its authorship and consequences to be called to question by those subject to algorithmic decision-making processes (James 2013; Diakopoulos 2013).”
Clarifying Journalism’s Quantitative Turn (2015, methodology): “Algorithms are the abstraction of a step-by-step procedure taking an input and producing an output to accomplish a defined outcome (Diakopoulos 2014a; Wing 2008).”
Auditing radicalization pathways on YouTube (2019, background): “ubt. Auditing the web. As algorithms play an ever-larger role in our lives, it is increasingly important for researchers and society at large to reverse engineer algorithms input-output relationships [15]. Previous large scale algorithmic auditing include measuring discrimination on AirBnB [16], per”
Accountable Artificial Intelligence: Holding Algorithms to Account (2020, background): “…reforms and individual firing decisions (O’Neil 2016), by fire departments to predict where fires will break out and to prioritize inspections (Diakopoulos 2014), in criminal justice (Angwin et al. 2016), and in a whole array of other areas as varied as healthcare (e.g., NHS collaborations…”
Breaking the filter bubble: democracy and design (2015, background): “This is because information can be prioritized, filtered and hidden depending on a user’s previous interaction with the system and other factors (Bozdag 2013; Diakopoulos 2014).”