COLD: Annotation scheme and evaluation data set for complex offensive language in English

By Alexis Palmer, Christine V. Carr, Melissa Robinson, Jordan Sanders. 2020

This paper presents a new, extensible annotation scheme for oﬀensive language data sets. The annotation scheme expands coverage beyond fairly straightforward cases of oﬀensive language to address several cases of complex, implicit, and/or pragmatically-triggered oﬀensive language. We apply the annotation scheme to create a new Complex Oﬀensive Language Data Set for English ( COLD-EN ). The primary purpose of this data set is to diagnose how well systems for automatic detection of abusive language are able to classify three types of complex oﬀensive language: reclaimed slurs, oﬀensive utterances containing pejorative adjectival nominalizations (and no slur terms), and utterances conveying oﬀense through linguistic distancing. COLD oﬀers a straightforward framework for error analysis. Our vision is that researchers will use this data set to diagnose the strengths and weaknesses of their oﬀensive language detection systems. In this paper, we diagnose some strengths and weaknesses of a top-performing oﬀensive language detection system by: a) using it to classify COLD , and b) investigating its performance on the 10 ﬁne-grained categories supported by our annotation scheme. We evaluate the system’s performance when trained on ﬁve diﬀerent standard data sets for oﬀensive language detection. Systems trained on diﬀerent data sets have diﬀerent strengths and weaknesses, with most performing poorly on the phenomena of reclaimed slurs and pejorative nominalizations. NOTE: This paper contains sensitive and oﬀensive material. The oﬀensive materials are part of a complex puzzle we wish to better understand; they appear in the form of lightly-censored slurs and degrading insults. We do not condone this type of language, nor does it reﬂect the attitudes or beliefs of the authors.

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

HateCheck: Functional Tests for Hate Speech Detection Models (2020, background): “Palmer et al. (2020) compile three datasets for evaluating model performance on what they call complex offensive language , speciﬁcally the use of reclaimed slurs, adjective nominalisation and linguistic distancing.”
Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection (2021, background): “Subtle and implicit forms of hate speech can also create false negatives (Vidgen and Yasseri, 2019; Palmer et al., 2020; Mathew et al., 2020), as well as more ‘complex’ forms of speech such as sarcasm, irony, adjective nominalization and rhetorical questions (Caselli et al., 2020; Vidgen et…”
Introducing CAD: the Contextual Abuse Dataset (2021, background): “Palmer et al. (2020) label implicit varieties of hate, including ‘adjectival nominalization’, ‘distancing’ and ‘Oth-ering’ language.”