BEEP! Korean Corpus of Online News Comments for Toxic Speech Detection
By Jihyung Moon, Won Ik Cho, Junbum Lee. 2020
108 citations recorded by the source corpus.
Toxic comments in online platforms are an unavoidable social issue under the cloak of anonymity. Hate speech detection has been actively done for languages such as English, German, or Italian, where manually labeled corpus has been released. In this work, we first present 9.4K manually labeled entertainment news comments for identifying Korean toxic speech, collected from a widely used online news platform in Korea. The comments are annotated regarding social bias and hate speech since both aspects are correlated. The inter-annotator agreement Krippendorff’s alpha score is 0.492 and 0.496, respectively. We provide benchmarks using CharCNN, BiLSTM, and BERT, where BERT achieves the highest score on all tasks. The models generally display better performance on bias identification, since the hate speech detection is a more subjective issue. Additionally, when BERT is trained with bias label for hate speech detection, the prediction score increases, implying that bias and hate are intertwined. We make our dataset publicly available and open competitions with the corpus and benchmarks.
Subfields: Artificial Intelligence, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yulia Tsvetkov
- Vidhisha Balachandran
- Sachin Kumar
- Antonios Anastasopoulos
- Lucille Njoo
- Mohit Bansal
- Anjalie Field
- Hinrich Schütze
In Natural Language Processing
- Yulia Tsvetkov
- Sachin Kumar
- Vidhisha Balachandran
- Antonios Anastasopoulos
- Lucille Njoo
- Anjalie Field
- Hinrich Schütze
- Mohit Bansal
Citations in the corpus
- Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey (2022, background): “Since language is inherently a tool of power—the primary means by *Equal contribution 1While the majority of these models are trained on English language datasets, recent studies have also obtained similar gains on other languages (Lin et al., 2021c; Shliazhko et al., 2022) which people and societie”