Thinking critically about and researching algorithms

By Rob Kitchin. 2014

823 citations recorded by the source corpus.

ABSTRACT More and more aspects of our everyday lives are being mediated, augmented, produced and regulated by software-enabled technologies. Software is fundamentally composed of algorithms: sets of defined steps structured to process instructions/data to produce an output. This paper synthesises and extends emerging critical thinking about algorithms and considers how best to research them in practice. Four main arguments are developed. First, there is a pressing need to focus critical and empirical attention on algorithms and the work that they do given their increasing importance in shaping social and economic life. Second, algorithms can be conceived in a number of ways – technically, computationally, mathematically, politically, culturally, economically, contextually, materially, philosophically, ethically – but are best understood as being contingent, ontogenetic and performative in nature, and embedded in wider socio-technical assemblages. Third, there are three main challenges that hinder research about algorithms (gaining access to their formulation; they are heterogeneous and embedded in wider systems; their work unfolds contextually and contingently), which require practical and epistemological attention. Fourth, the constitution and work of algorithms can be empirically studied in a number of ways, each of which has strengths and weaknesses that need to be systematically evaluated. Six methodological approaches designed to produce insights into the nature and work of algorithms are critically appraised. It is contended that these methods are best used in combination in order to help overcome epistemological and practical challenges.

Subfields: Computer Science Applications, Safety Research, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

In Safety Research

In Sociology and Political Science

Citations in the corpus

Addressing Age-Related Bias in Sentiment Analysis (2018, background): “As part of the larger discussion of algorithmic bias, recent work has begun to analyze the design and underlying mechanisms of algorithms that contribute to bias, with a call for more empirical studies [42].”
Truth from the machine: artificial intelligence and the materialization of identity (2021, background): “Even after coming up with a formal definition, systems can be thought about ‘technically, computationally, mathematically, politically, culturally, economically, contextually, materially, philosophically, ethically, and so on’ (Kitchin 2017).”
Managing visibility on YouTube through algorithmic gossip (2019, methodology): “Such an approach can be seen as an informal take on the research technique of ‘reverse engineering’, which is defined as evaluating algorithmic input and output to determine how the ‘recipe of the algorithm is composed’ (Kitchin, 2017: 24).”
Toward Fair, Accountable, and Transparent Algorithms: Case Studies on Algorithm Initiatives in Korea and China (2019, background): “Algorithm systems are composed of one or more technological algorithms, and at the same time an algorithm re ﬂ ects users ’ knowledge and social experiences and ultimately user acceptance (Shin 2014; Kitchin 2017).”
Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management (2018, background): “Our focus of inquiry—everyday people’s perceptions of algorithms—is closely related to recent discourses around socio-technical deﬁnitions of algorithms (Gillespie, 2014; Kitchin, 2016), which go beyond the technical, entitative deﬁnition described above.”
Experiencing Algorithms: How Young People Understand, Feel About, and Engage With Algorithmic News Selection on Social Media (2021, background): “These actions help to shape and refine algorithms of personalized news media, as feed-back loops between user behavior and future content exposure emerge (Kitchin, 2017; Thorson, 2020).”
WeBuildAI (2019, background): “This direct approach can minimize potential errors and biases that occur when codifying policy ideas into computational algorithms, which has been highlighted as a risk in algorithmic governance [45].”
Between Subjectivity and Imposition (2020, background): “Moreover, critical research has argued that data-driven systems embody the personal and corporate values and interests of the people and organizations involved in their development [31, 51, 53, 57].”
Anxiety, panic and self-optimization (2018, methodology): “Recommended videos are selected by YouTube’s ‘black-boxed’ algorithm, so called because YouTube ‘provides little explanatory insight into the relative influence of the independent variables in the process’ (Olden and Jackson, 2002: 135). and researchers (Kitchin, 2016).”
Transparent to whom? No algorithmic accountability without a critical audience (2018, background): “Qualitative approaches can help draw detailed and rich accounts of algorithmic model use (Kitchin, 2017).”
ORES (2019, background): “stems in their social spaces has typically been more of a focus from the Critical Algorithms Studies literature, which comes from a more humanistic and interpretivist social science perspective (e.g. [6, 59]). This literature also emphasizes a need to see algorithmic systems as dynamic and constantl”
In AI we trust? Perceptions about automated decision-making by artificial intelligence (2020, background): “They are “cre-ated for purposes that are often far from neutral: to create value and capital; to nudge behavior and structure preferences in a certain way; and to identify, sort and classify people” (Kitchin 2017, p. 18).”
Do not blame it on the algorithm: an empirical assessment of multiple recommender systems and their impact on content diversity (2018, neutral): “In order to understand the effects of a recommendation algorithm on individual ’ s news diet, it is important to understand that potential effects of recommendation algorithms are dependent on the exact design of the recommendation algorithm in use (see also Kitchin, 2017).”