Protest Activity Detection and Perceived Violence Estimation from Social Media Images

By Donghyeon Won, Zachary C. Steinert-Threlkeld, Jungseock Joo. 2017

104 citations recorded by the source corpus.

We develop a novel visual model which can recognize protesters, describe their activities by visual attributes and estimate the level of perceived violence in an image. Studies of social media and protests use natural language processing to track how individuals use hashtags and links, often with a focus on those items' diffusion. These approaches, however, may not be effective in fully characterizing actual real-world protests (e.g., violent or peaceful) or estimating the demographics of participants (e.g., age, gender, and race) and their emotions. Our system characterizes protests along these dimensions. We have collected geotagged tweets and their images from 2013-2017 and analyzed multiple major protest events in that period. A multi-task convolutional neural network is employed in order to automatically classify the presence of protesters in an image and predict its visual attributes, perceived violence and exhibited emotions. We also release the UCLA Protest Image Dataset, our novel dataset of 40,764 images (11,659 protest images and hard negatives) with various annotations of visual attributes and sentiments. Using this dataset, we train our model and demonstrate its effectiveness. We also present experimental results from various analysis on geotagged image data in several prevalent protest events. Our dataset will be made accessible at https://www.sscnet.ucla.edu/comm/jjoo/mm-protest/.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Sociology and Political Science, Social Sciences & Humanities

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Sociology and Political Science

In Social Sciences & Humanities

Citations in the corpus

FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age (2019, methodology): “Notable examples are demographic analyses of social media users using their photographs [9, 43, 64, 65, 62].”
Understanding the Political Ideology of Legislators from Social Media Images (2019, background): “Recently, scholars have adopted computer vision approaches and large scale visual data in social media for research projects in social science and media analysis (Ha et al., 2018, Won, Steinert-Threlkeld, and Joo, 2017, Zhang and Pan, 2019).”
FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age for Bias Measurement and Mitigation (2021, methodology): “No-table examples are demographic analyses of social media users using their photographs [9, 45, 65, 66, 63].”