Reporting, Reviewing, and Responding to Harassment on Twitter

By J. N. Matias, Amy Johnson, Whitney Erin Boesel, Brian Keegan, Jaclyn Friedman, Charles DeTar. 2015

94 citations recorded by the source corpus.

When people experience harassment online, from individual threats or invective to coordinated campaigns of harassment, they have the option to report the harassers and content to the platform where the harassment has occurred. Platforms then evaluate harassment reports against terms of use and other policies to decide whether to remove content or take action against the alleged harasser--or not. On Twitter, harassing accounts can be deleted entirely, suspended (with content made unavailable pending appeal or specific changes), or sent a warning. Some platforms, including Twitter and YouTube, grant authorized reporters or trusted flaggers special privileges to identify and report inappropriate content on behalf of others. In November 2014, Twitter granted Women, Action, and the Media (WAM!) this authorized reporter status. In three weeks, WAM! reviewers assessed 811 incoming reports of harassment and escalated 161 reports to Twitter, ultimately seeing Twitter carry out 70 account suspensions, 18 warnings, and one deleted account. This document presents findings from this three-week project; it draws on both quantitative and qualitative methods. Findings focus on the people reporting and receiving harassment, the kinds of harassment that were reported, Twitter's response to harassment reports, the process of reviewing harassment reports, and challenges for harassment reporting processes.

Subfields: Artificial Intelligence, Communication

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Communication

Citations in the corpus

Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019, background): “…may not include people who only read content without writing), or due to self-censorship (e.g., not sharing or deleting a post) as a result of factors such as online harassment, privacy concerns, or some sort of social repercussion (Wang et al., 2011; Das and Kramer, 2013; Matias et al., 2015).”
The Effect of Extremist Violence on Hateful Speech Online (2018, background): “Finally, our analysis is retrospective, and harmful content is actively deleted by many social media platforms when re-ported (Matias et al. 2015), which can result in incomplete data collections.”
Community Interaction and Conflict on the Web (2018, background): “Instances of one online community harassing, “brigading”, or “trolling” another will inevitably occur [23, 28, 51], but how can communities defend against these”
Contestability in Algorithmic Systems (2019, background): “Interest has also grown in contesting decision making processes at scale, often in the form of auditing and detecting bias in these systems (for example, collectively auditing Twitter’s content moderation [9]).”
When Are Search Completion Suggestions Problematic? (2020, methodology): “For the remaining of our analysis, we consider a <prefix> [<suffix>] pair problematic if at least one judge marked it as such—following a reporting model used in applications where user complaint reports are reviewed irrespective of how many users submit them [14, 48].”
Online Harassment and Content Moderation (2018, background): “e problem of online harassment is particularly prevalent on Twier1 [20, 31].”