CrisisLex: A Lexicon for Collecting and Filtering Microblogged Communications in Crises

By Alexandra Olteanu, Carlos Castillo, Fernando Diaz, Sarah Vieweg. 2014

450 citations recorded by the source corpus.

Locating timely, useful information during crises and mass emergencies is critical for those forced to make potentially life-altering decisions. As the use of Twitter to broadcast useful information during such situations becomes more widespread, the problem of finding it becomes more difficult. We describe an approach toward improving the recall in the sampling of Twitter communications that can lead to greater situational awareness during crisis situations. First, we create a lexicon of crisis-related terms that frequently appear in relevant messages posted during different types of crisis situations. Next, we demonstrate how we use the lexicon to automatically identify new terms that describe a given crisis. Finally, we explain how to efficiently query Twitter to extract crisis-related messages during emergency events. In our experiments, using a crisis lexicon leads to substantial improvements in terms of recall when added to a set of crisis-specific keywords manually chosen by experts; it also helps to preserve the original distribution of message types.

Subfields: Communication, Sociology and Political Science, Statistical and Nonlinear Physics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Communication

In Sociology and Political Science

In Statistical and Nonlinear Physics

Citations in the corpus

Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019, background): “2014], by expanding and adapting user queries [Magdy and Elsayed 2014], by exploiting domain patterns for query generation and expansion [Olteanu et al. 2014a], by leveraging recent seen content to maintain a uniform sampling [Osborne et al.”
A Critical Review of Online Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2018, background): “We will review both recent work raising concerns about what can be legitimately inferred from existing social datasets [31, 35], as well as research that tries to quantify the limits of such data [10, 14, 19, 27].”
CrisisMMD: Multimodal Twitter Datasets from Natural Disasters (2018, background): “The dataset is publicly available at CrisisLex site 3 .”
The Effect of Extremist Violence on Hateful Speech Online (2018, methodology): “This method is known to improve the coverage of social media data (Olteanu et al. 2014; Davidson et al. 2017).”
Processing Social Media Messages in Mass Emergency (2014, methodology): “…2013] By information provided Caution and advice caution & advice [Imran et al. 2013b]; warnings [Acar and Muraki 2011]; hazard, preparation [Olteanu et al. 2014]; tips [Leavitt and Clark 2014]; advice [Bruns et al. 2012]; status, protocol [Hughes et al. 2014b] Affected people people…”
Tackling Climate Change with Machine Learning (2019, background): “ML can help properly surface these insights, compressing large volumes of social media data into the key takeaways, which can be acted upon by disaster managers [624, 639, 640].”
When Are Search Completion Suggestions Problematic? (2020, methodology): “The sets of key-phrases were assembled following standard practices in keyword-centered data collections [10, 28, 58], including phrases used in [10, 16] or specific to topics and categories mentioned in prior work (e.”
Comparing Events Coverage in Online News and Social Media: The Case of Climate Change (2015, background): “year, park, hell, light (Olteanu et al. 2014), which we manually filter out.”