Combining Search, Social Media, and Traditional Data Sources to Improve Influenza Surveillance

By Mauricio Santillana, Anh Gia-Tuan Nguyen, Mark Dredze, Michael J. Paul, Elaine O. Nsoesie, John S. Brownstein. PLoS Comput. Biol., 2015

426 citations recorded by the source corpus.

We present a machine learning-based methodology capable of providing real-time (“nowcast”) and forecast estimates of influenza activity in the US by leveraging data from multiple data sources including: Google searches, Twitter microblogs, nearly real-time hospital visit records, and data from a participatory surveillance system. Our main contribution consists of combining multiple influenza-like illnesses (ILI) activity estimates, generated independently with each data source, into a single prediction of ILI utilizing machine learning ensemble approaches. Our methodology exploits the information in each data source and produces accurate weekly ILI predictions for up to four weeks ahead of the release of CDC’s ILI reports. We evaluate the predictive ability of our ensemble approach during the 2013–2014 (retrospective) and 2014–2015 (live) flu seasons for each of the four weekly time horizons. Our ensemble approach demonstrates several advantages: (1) our ensemble method’s predictions outperform every prediction using each data source independently, (2) our methodology can produce predictions one week ahead of GFT’s real-time estimates with comparable accuracy, and (3) our two and three week forecast estimates have comparable accuracy to real-time predictions using an autoregressive model. Moreover, our results show that considerable insight is gained from incorporating disparate data streams, in the form of social media and crowd sourced data, into influenza predictions in all time horizons.

Subfields: Epidemiology

View Paper: https://www.semanticscholar.org/paper/84fed2a181f282fde8251882ea2a3f0b1e65bbe5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Epidemiology

1. Nicola Perra
2. Marcel Salathé

Citations in the corpus

• Predicting COVID-19 Incidence Through Analysis of Google Trends Data in Iran: Data Mining and Deep Learning Pilot Study (2020, background): “Past work on influenza and Zika virus predictions, for example, in the study by Santillana et al [13] in 2015, proposed a machine learning method for predicting influenza in the United States.”
• Accurate estimation of influenza epidemics using Google search data via ARGO (2015, background): “Further improvements in influenza prediction may come from combining multiple predictors constructed from disparate data sources (40).”
• Statistical physics of vaccination (2016, background): “The first platform engages individuals living in USA or Canada.”