Limits of Electoral Predictions Using Twitter

By Daniel Gayo-Avello, Panagiotis Takis Metaxas, Eni Mustafaraj. 2011

302 citations recorded by the source corpus.

Using social media for political discourse is becoming common practice, especially around election time. One interesting aspect of this trend is the possibility of pulsing the public’s opinion about the elections, and that has attracted the interest of many researchers and the press. Allegedly, predicting electoral outcomes from social media data can be feasible and even simple. Positive results have been reported, but without an analysis on what principle enables them. Our work puts to test the purported predictive power of socialmedia metrics against the 2010 US congressional elections. Here, we applied techniques that had reportedly led to positive election predictions in the past, on the Twitter data collected from the 2010 US congressional elections. Unfortunately, we find no correlation between the analysis results and the electoral outcomes, contradicting previous reports. Observing that 80 years of polling research would support our findings, we argue that one should not be accepting predictions about events using social media data as a black box. Instead, scholarly research should be accompanied by a model explaining the predictive power of social media, when there is one.

Subfields: Epidemiology, Sociology and Political Science

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Epidemiology

In Sociology and Political Science

Citations in the corpus

Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019, background): “, 2015b; Pavalanathan and Eisenstein, 2015), or exit pools or election results (Gayo-Avello et al., 2011; Gayo-Avello, 2012; Gayo-Avello, 2013).”
Sentiment analysis on evolving social streams: how self-report imbalances can help (2014, background): “Although such reporting bias is usually perceived as a source of inaccuracy [28, 16], here we argue that the self-reporting nature of social media, when observed on large-scale social network data, may actually provide signals that ease the task of sentiment tracking in online environments, provided”
Air pollution as a cause of sleeplessness: Social media evidence from a panel of Chinese cities (2019, background): “5 (1) (2) Average Humidity (H ∈[40,60) Omitted) H <20 -42.”
Early Prediction of Movie Box Office Success Based on Wikipedia Activity Big Data (2012, background): “Another example is using Twitter to predict electoral outcomes [5], however with its biases and limitations [6,7].”
Correlating financial time series with micro-blogging activity (2012, background): “[11] have similar objections, and in addition show evidence of differences between the distribution of demographic characteristics of Twitter users from a country compared to the general population of that country.”
On Using Twitter to Monitor Political Sentiment and Predict Election Results (2011, background): “(Gayo-Avello et al., 2011) They cite the two primary caveats with using social media to inform predictive models as selection bias (inability to determine a representative sample) and potential for deliberate inﬂuence of results (through gaming and spamming for example).”
External validity of sentiment mining reports: Can current methods identify demographic biases, event biases, and manipulation of reviews? (2014, methodology): “Gayo-Avello [16,17] argues that skewness in demographics contributes to failures”
Quantifying Political Leaning from Tweets and Retweets (2013, background): “…an event time interval are necessarily talking about the event, and (c) political tweets are difficult to classify (Maynard and Funk 2011; Mejova, Srinivasan, and Boynton 2013), and off-the-shelf sentiment analysis tools may not perform sufficiently well (Gayo-Avello, Metaxas, and Mustafaraj 2011).”
Predicting the 2011 Dutch Senate Election Results with Twitter (2012, background): “Gayo-Avello et al. (2011) show that predictions based on Twitter only predicted half of the winners of US congressional elections with two candidates correctly, a performance which is not better than chance.”