On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Research

By Luiza Amador Pozzobon, B. Ermiş, Patrick Lewis, Sara Hooker. 2023

59 citations recorded by the source corpus.

Perception of toxicity evolves over time and often differs between geographies and cultural backgrounds. Similarly, black-box commercially available APIs for detecting toxicity, such as the Perspective API, are not static, but frequently retrained to address any unattended weaknesses and biases. We evaluate the implications of these changes on the reproducibility of findings that compare the relative merits of models and methods that aim to curb toxicity. Our findings suggest that research that relied on inherited automatic toxicity scores to compare models and techniques may have resulted in inaccurate findings. Rescoring all models from HELM, a widely respected living benchmark, for toxicity with the recent version of the API led to a different ranking of widely used foundation models. We suggest caution in applying apples-to-apples comparisons between studies and lay recommendations for a more structured approach to evaluating toxicity over time. Code and data are available at https://github.com/for-ai/black-box-api-challenges.

Subfields: Artificial Intelligence, Computational Theory and Mathematics, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

In Safety, Robustness & Interpretability

Citations in the corpus

Evaluating the Social Impact of Generative AI Systems in Systems and Society (2023, methodology): “For evaluations which rely on a third party API, such as the many benchmarks which leverage Google Perspective API [182] for toxicity detection, it is important to make sure comparisons between models are standardized using the same version of the API to avoid reproducibility issues [185].”
Using In-Context Learning to Improve Dialogue Safety (2023, methodology): “As re-producibility concerns have been raised about Per-spectiveAPI (Pozzobon et al., 2023), we use C LAS - SIFIER as our primary tool for evaluating safety.”
PaLM 2 Technical Report (2023, methodology): “We also used a fixed version of the Perspective API for all evaluations (Pozzobon et al., 2023), and so evaluations are not taking advantage of performance improvements in toxicity classification.”
Bias and Fairness in Large Language Models: A Survey (2023, background): “Qian et al. (2019), for instance, propose an equalizing objective that encourages demographic words to be predicted with equal probability.”
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models (2023, background): “Uncontrolled Versioning : Proprietary models undergo version updates that are often beyond the users’ purview or control (Pozzobon et al., 2023).”