Rayyan—a web and mobile app for systematic reviews
By Mourad Ouzzani, Hossam M. Hammady, Z. Fedorowicz, Ahmed K. Elmagarmid. Systematic Reviews, 2016
21,142 citations recorded by the source corpus.
BackgroundSynthesis of multiple randomized controlled trials (RCTs) in a systematic review can summarize the effects of individual outcomes and provide numerical answers about the effectiveness of interventions. Filtering of searches is time consuming, and no single method fulfills the principal requirements of speed with accuracy. Automation of systematic reviews is driven by a necessity to expedite the availability of current best evidence for policy and clinical decision-making.We developed Rayyan (http://rayyan.qcri.org), a free web and mobile app, that helps expedite the initial screening of abstracts and titles using a process of semi-automation while incorporating a high level of usability. For the beta testing phase, we used two published Cochrane reviews in which included studies had been selected manually. Their searches, with 1030 records and 273 records, were uploaded to Rayyan. Different features of Rayyan were tested using these two reviews. We also conducted a survey of Rayyan’s users and collected feedback through a built-in feature.ResultsPilot testing of Rayyan focused on usability, accuracy against manual methods, and the added value of the prediction feature. The “taster” review (273 records) allowed a quick overview of Rayyan for early comments on usability. The second review (1030 records) required several iterations to identify the previously identified 11 trials. The “suggestions” and “hints,” based on the “prediction model,” appeared as testing progressed beyond five included studies. Post rollout user experiences and a reflexive response by the developers enabled real-time modifications and improvements. The survey respondents reported 40% average time savings when using Rayyan compared to others tools, with 34% of the respondents reporting more than 50% time savings. In addition, around 75% of the respondents mentioned that screening and labeling studies as well as collaborating on reviews to be the two most important features of Rayyan.As of November 2016, Rayyan users exceed 2000 from over 60 countries conducting hundreds of reviews totaling more than 1.6M citations. Feedback from users, obtained mostly through the app web site and a recent survey, has highlighted the ease in exploration of searches, the time saved, and simplicity in sharing and comparing include-exclude decisions. The strongest features of the app, identified and reported in user feedback, were its ability to help in screening and collaboration as well as the time savings it affords to users.ConclusionsRayyan is responsive and intuitive in use with significant potential to lighten the load of reviewers.
Subfields: General Health Professions, Sociology and Political Science, Statistics and Probability
View Paper: https://www.semanticscholar.org/paper/1227f2fefc26f204d2ce07b005aedd7d74e6e72a
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In General Health Professions
- Matthew J. Page
- J. Gurevitch
- J. Koricheva
- Shinichi Nakagawa
- David Moher
- M. Lundqvist
- Stephen M. Schueller
- Fredrik Bondestam
In Sociology and Political Science
- Matthew J. Page
- J. Gurevitch
- J. Koricheva
- Shinichi Nakagawa
- David Moher
- M. Lundqvist
- G. Stewart
- Michael D. Jennions
In Statistics and Probability
- Matthew J. Page
- J. Koricheva
- J. Gurevitch
- M. Lagisz
- David Moher
- Shinichi Nakagawa
- G. Stewart
- R. O’Dea
Citations in the corpus
- Mental health outcomes of quarantine and isolation for infection prevention: a systematic umbrella review of the global evidence (2020, methodology): “Further, these citations were exported to Rayyan [38], which is a cloud-based platform for screening citations data.”
- Towards Automated Data Cleaning Workflows (2019, background): “…anecdotally show that the approach is promising, we evaluate our performance estimator on 11 diverse datasets: Hospital [25], Flights [25], Rayyan [20], IT [1], Beers [10] are our real-world datasets that have been cleaned manually, and Salaries [29], Address [1], Movies [7], Restaurants [7],…”
- Sexual harassment in higher education – a systematic review (2020, methodology): “were registered in a free beta-version of the online tool Rayyan QCR1 (Ouzzani et al. 2016) including title, abstract, authors, keywords and several other criteria.”
- Preferred reporting items for systematic reviews and meta‐analyses in ecology and evolutionary biology: a PRISMA extension (2021, methodology): “Software can help speed up the process of title and abstract screening (e.g. Rathbone, Hoffmann & Glasziou, 2015b; Ouzzani et al., 2016).”
- Characterising long COVID: a living systematic review (2021, methodology): “19 Data extraction Data extraction was performed using Microsoft Excel.”
- A practical guide to question formation, systematic searching and study screening for literature reviews in ecology and evolution (2021, methodology): “These duplicates can be removed with reference managers, literature- screening tools such as Rayyan (Ouzzani et al., 2016), or R packages that use simple matching or fuzzy logic (Westgate, 2019).”
- Gamification of health professions education: a systematic review (2020, methodology): “After retrieving the search results from different databases, AvG removed the duplicates and uploaded the remaining articles to Rayyan, a mobile and web-based application developed for systematic reviews (Ouzzani et al. 2016).”
- Digital Mental Health Interventions for Depression, Anxiety, and Enhancement of Psychological Well-Being Among College Students: Systematic Review (2018, positive): “Search results were uploaded into Rayyan, a Web-based software program that allows for reviewers to collaborate during the study selection process [27].”