Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact

By Gunther Eysenbach. Journal of Medical Internet Research, 2011

1,054 citations recorded by the source corpus.

Background Citations in peer-reviewed articles and the impact factor are generally accepted measures of scientific impact. Web 2.0 tools such as Twitter, blogs or social bookmarking tools provide the possibility to construct innovative article-level or journal-level metrics to gauge impact and influence. However, the relationship of the these new metrics to traditional metrics such as citations is not known. Objective (1) To explore the feasibility of measuring social impact of and public attention to scholarly articles by analyzing buzz in social media, (2) to explore the dynamics, content, and timing of tweets relative to the publication of a scholarly article, and (3) to explore whether these metrics are sensitive and specific enough to predict highly cited articles. Methods Between July 2008 and November 2011, all tweets containing links to articles in the Journal of Medical Internet Research (JMIR) were mined. For a subset of 1573 tweets about 55 articles published between issues 3/2009 and 2/2010, different metrics of social media impact were calculated and compared against subsequent citation data from Scopus and Google Scholar 17 to 29 months later. A heuristic to predict the top-cited articles in each issue through tweet metrics was validated. Results A total of 4208 tweets cited 286 distinct JMIR articles. The distribution of tweets over the first 30 days after article publication followed a power law (Zipf, Bradford, or Pareto distribution), with most tweets sent on the day when an article was published (1458/3318, 43.94% of all tweets in a 60-day period) or on the following day (528/3318, 15.9%), followed by a rapid decay. The Pearson correlations between tweetations and citations were moderate and statistically significant, with correlation coefficients ranging from .42 to .72 for the log-transformed Google Scholar citations, but were less clear for Scopus citations and rank correlations. A linear multivariate model with time and tweets as significant predictors (P < .001) could explain 27% of the variation of citations. Highly tweeted articles were 11 times more likely to be highly cited than less-tweeted articles (9/12 or 75% of highly tweeted article were highly cited, while only 3/43 or 7% of less-tweeted articles were highly cited; rate ratio 0.75/0.07 = 10.75, 95% confidence interval, 3.4–33.6). Top-cited articles can be predicted from top-tweeted articles with 93% specificity and 75% sensitivity. Conclusions Tweets can predict highly cited articles within the first 3 days of article publication. Social media activity either increases citations or reflects the underlying qualities of the article that also predict citations, but the true use of these metrics is to measure the distinct concept of social impact. Social impact measures based on tweets are proposed to complement traditional citation metrics. The proposed twimpact factor may be a useful and timely metric to measure uptake of research findings and to filter research findings resonating with the public in real time.

Subfields: Health, Information Systems and Management, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/0a1e14802ba4f5513d5954d6e1475e8503e6aca4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Health

In Information Systems and Management

In Statistics and Probability

Citations in the corpus

Twitter (2020, neutral): “tweets, median (IQR) 0 (0-1) 2 (2-10) 3 (1-9) 17 (3-46) 2 (0-7) < 0.”
Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics (2014, background): “ about the intentional, motivational or experiential motives of the users” (p. 19). Many of the studies published so far have merely calculated correlations between citations and altmetrics (see e.g. Eysenbach, 2011). However, little knowledge is gained from these studies. The correlation on a mediu”
Do Altmetrics Work? Twitter and Ten Other Social Web Services (2013, background): “Finally, unlike one previous study [24], no predictive power can be claimed from the results.”
How the Scientific Community Reacts to Newly Submitted Preprints: Article Downloads, Twitter Mentions, and Citations (2012, background): “016) implying that the observed correlation is strongest when frequently mentioned articles on Twitter are included, matching the results reported by [6].”
Scientists on Twitter: Preaching to the choir or singing from the rooftops? (2018, background): “For example, papers that are tweeted about more often also accumulate more citations (Eysenbach 2011; Thelwall et al. 2013; Peoples et al. 2016), and the volume of tweets in the first week following publication correlates with the likelihood of a paper becoming highly cited (Eysenbach 2011), althoug”
Scholarly use of social media and altmetrics: A review of the literature (2016, background): “As one might expect from this medium, the “twitter window” of scientific papers is quite short, with an increase following publication and fast decay (Eysenbach, 2011; Shuai et al., 2012).”
An Introduction to Social Media for Scientists (2013, background): “However, Eysenbach [7] reported that highly tweeted journal articles were 11 times more likely to be highly cited versus articles without strong social media coverage.”
A review of theory and practice in scientometrics (2015, background): “04) found a weak correlation with citations. Downloaded/Saved: as viewed plus CiteUlike, Mendelay. Discussed: Wikipedia, Facebook, Twitter, Natureblogs28, ScienceSeeker29, general research blogs. Eysenbach (2011) suggested a “twimpact factor” based on the number of tweets Recommended: F1000Prime3”
Social Media: A Review and Tutorial of Applications in Medicine and Health Care (2014, background): “A number of indicators suggest that the evidence for using social media in the health care context is growing; for example, the number of articles indexed on PubMed has nearly doubled each year for the last 4 years [12], social media policies are being adopted [13] and tested in various health care ”