The Limits of Abstract Evaluation Metrics: The Case of Hate Speech Detection

By Alexandra Olteanu, Kartik Talamadupula, Kush R. Varshney. 2017

44 citations recorded by the source corpus.

Wagstaff (2012) draws attention to the pervasiveness of abstract evaluation metrics that explicitly ignore or remove problem specifics. While such metrics allow practitioners to compare numbers across application domains, they offer limited insight into the impact of algorithmic decisions on humans and their perception of the algorithm's correctness. Even for problems that are mathematically the same, both the real-cost of (mathematically) identical errors, as well as their perceived-cost by users, may significantly vary according to the specifics of each problem domain, as well as of the user perceiving the result. While the real-cost of errors has been considered previously, little attention has been paid to the perceived-cost issue. We advocate for the inclusion of human-centered metrics that elicit error costs from humans from two perspectives: the nature of the error, and the user context. Focusing on hate speech detection on social media, we demonstrate that even when fixing the performance as measured by an abstract metric such as precision, user perception of correctness varies greatly depending on the nature of errors and user characteristics.

Subfields: Artificial Intelligence, Information Systems, Sociology and Political Science, Natural Language Processing, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Sociology and Political Science

In Natural Language Processing

In Safety, Robustness & Interpretability

Citations in the corpus

Language (Technology) is Power: A Critical Survey of “Bias” in NLP (2020, background): “What are the impacts of evaluation metrics (Olteanu et al., 2017)?”
“I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset (2022, background): “Adopting a general definition of bias as difference allows for NLP practitioners to make the delineation between benign and harmful for each identity term separately, based on the particular task and use case at hand (Olteanu et al., 2017; Blodgett et al., 2020; Czarnowska et al., 2021; Dev et al., ”
On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products (2016, background): “ution is a discussion on how to even understand and quantify the desirability and undesirability of outcomes along with their costs. To complement simply eliciting such knowledge directly from people [11], we suggest a data-driven approach for characterizing harms that are particularly relevant for ”
Increasing Trust in AI Services through Supplier's Declarations of Conformity (2018, background): “Abstract loss functions may be informed by real-world quality metrics [31], including context-dependent ones [32].”
Your fairness may vary: Pretrained language model fairness in toxic text classification (2021, background): “We also recognize that abstract metrics have limitations and the societal impacts resulting from bias mitigation are not well understood (Olteanu et al., 2017).”
FactSheets: Increasing Trust in AI Services through Supplier's Declarations of Conformity. (2018, background): “Abstract loss functions may be informed by real-world quality metrics [36], including context-dependent ones [37].”
A Critical Review of Online Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2018, background): “In addition, we discuss existing solutions aiming at correcting or accounting for biased or innacurate datasets, and existing recommendations for adequately evaluate and methods and observations (including the need for disclaimers and standards [6, 8]), or to interpret results in context [28, 30].”
Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking (2021, background): “There is no single, best way to measure model fairness [35, 50, 3].”
The Effect of Extremist Violence on Hateful Speech Online (2018, background): “Ex-pressions of hate take many forms (Ghanea 2013; Matias et al. 2015); they can be ambiguous, and the perception of what is hateful varies between individuals (Olteanu et al. 2017).”
FactSheets: Increasing trust in AI services through supplier's declarations of conformity (2019, background): “Abstract loss functions may be informed by real-world quality metrics [36], including context-dependent ones [37].”
Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics (2021, background): “In line with the recommendations of Olteanu et al. (2017) and Blodgett et al. (2020), we assert that fairness metrics need to be grounded in the application domain and carefully matched to the type of studied bias to offer meaningful insights.”