Improving Hate Speech Detection with Deep Learning Ensembles
By Steven Zimmerman, Udo Kruschwitz, Chris Fox. 2018
132 citations recorded by the source corpus.
Hate speech has become a major issue that is currently a hot topic in the domain of social media. Simultaneously, current proposed methods to address the issue raise concerns about censorship. Broadly speaking, our research focus is the area human rights, including the development of new methods to identify and better address discrimination while protecting freedom of expression. As neural network approaches are becoming state of the art for text classification problems, an ensemble method is adapted for usage with neural networks and is presented to better classify hate speech. Our method utilizes a publicly available embedding model, which is tested against a hate speech corpus from Twitter. To confirm robustness of our results, we additionally test against a popular sentiment dataset. Given our goal, we are pleased that our method has a nearly 5 point improvement in F-measure when compared to original work on a publicly available hate speech evaluation dataset. We also note difficulties encountered with reproducibility of deep learning methods and comparison of findings from other work. Based on our experience, more details are needed in published work reliant on deep learning methods, with additional evaluation information a consideration too. This information is provided to foster discussion within the research community for future work.
Subfields: Artificial Intelligence, Communication, Language and Linguistics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sean MacAvaney
- Nazli Goharian
- Ophir Frieder
- Michael J. Paul
- Xiaolei Huang
- Eugene Yang
- Linzi Xing
- Katina Russell
In Communication
- Michael J. Paul
- Xiaolei Huang
- Eugene Yang
- Linzi Xing
- Nazli Goharian
- Ophir Frieder
- Katina Russell
- Sean MacAvaney
In Language and Linguistics
Citations in the corpus
- Challenges for Toxic Comment Classification: An In-Depth Error Analysis (2018, neutral): “Zimmerman et al. (2018) investigated ensembling models with different hyper-parameters.”
- Multilingual Sentiment Analysis and Toxicity Detection for Text Messages in Russian (2021, background): “Up to this point, the identification of hateful and offensive statements was performed within the framework of solving the problem of detecting and classifying hate speech [7], work on which continues to this day [4], [8], [9].”
- Hate speech detection: Challenges and solutions (2019, background): “propose an ensemble approach, which combines the decisions of ten convolutional neural networks with different weight initializations [10].”
- Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition (2020, methodology): “We implement the Convolutional Neural Network (CNN) classifier described in (Kim, 2014; Zimmerman et al., 2018) by Keras (Chollet and others, 2015).”