A New Generation of Perspective API: Efficient Multilingual Character-level Transformers

By Alyssa Lees, Vinh Q. Tran, Yi Tay, Jeffrey Scott Sorensen, Jai Gupta, Donald Metzler, Lucy Vasserman. 2022

306 citations recorded by the source corpus.

On the world wide web, toxic content detectors are a crucial line of defense against potentially hateful and offensive messages. As such, building highly effective classifiers that enable a safer internet is an important research area. Moreover, the web is a highly multilingual, cross-cultural community that develops its own lingo over time. As such, it is crucial to develop models that are effective across a diverse range of languages, usages, and styles. In this paper, we present the fundamentals behind the next version of the Perspective API from Google Jigsaw. At the heart of the approach is a single multilingual token-free Charformer model that is applicable across a range of languages, domains, and tasks. We demonstrate that by forgoing static vocabularies, we gain flexibility across a variety of settings. We additionally outline the techniques employed to make such a byte-level model efficient and feasible for productionization. Through extensive experiments on multilingual toxic comment classification benchmarks derived from real API traffic and evaluation on an array of code-switching, covert toxicity, emoji-based hate, human-readable obfuscation, distribution shift, and bias evaluation settings, we show that our proposed approach outperforms strong baselines. Finally, we present our findings from deploying this system in production.

Subfields: Artificial Intelligence, Model Architectures, AI Ethics, Fairness & Society, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Model Architectures

In Signal Processing

Citations in the corpus

Unifying Language Learning Paradigms (2022, background): “For instance, regression is the probably only reason why one would add a task speciﬁc head (Lees et al., 2022), or to squeeze out some efﬁciency gains from eliminating a full vocabulary.”
WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs (2024, background): “There exists an extensive body of work on detecting hateful, toxic, offensive and abusive content [12, 29, 21] on online social networking sites such as Twitter [38] and Reddit [14].”
Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations (2023, methodology): “Lastly, all available tools use conventional transformer models that are small in size as their backbone (Markov et al., 2023; Lees et al., 2022).”
A Holistic Approach to Undesired Content Detection in the Real World (2022, unknown): “There is a long track record of work on the definition and detection of hateful, toxic, offensive and abusive content (Kwok and Wang 2013; Nobata et al. 2016; Waseem 2016; de Gibert et al. 2018; Vidgen et al. 2019; Gehman et al. 2020; Rosenthal et al. 2020; Lees et al. 2022).”
Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (2023, unknown): “Though, as we mention, closed models have more options for mitigations, the popularization of customization via fine-tuning APIs does bring the risk profile of closed-access models closer to that of open-access models.”
PaLI-X: On Scaling up a Multilingual Vision and Language Model (2023, methodology): “We generate captions and use the Perspective API [86] (threshold > 0.”