Guidelines for snowballing in systematic literature studies and a replication in software engineering

By Claes Wohlin. 2014

4,216 citations recorded by the source corpus.

Background: Systematic literature studies have become common in software engineering, and hence it is important to understand how to conduct them efficiently and reliably. Objective: This paper presents guidelines for conducting literature reviews using a snowballing approach, and they are illustrated and evaluated by replicating a published systematic literature review. Method: The guidelines are based on the experience from conducting several systematic literature reviews and experimenting with different approaches. Results: The guidelines for using snowballing as a way to search for relevant literature was successfully applied to a systematic literature review. Conclusions: It is concluded that using snowballing, as a first search strategy, may very well be a good alternative to the use of database searches.

Subfields: Computer Networks and Communications, Information Systems

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

Citations in the corpus

Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) (2018, methodology): “The list of selected papers was largely completed afterwards, by using a backward and forward snowballing strategy that consists of using the reference list of the selected papers and the citations to these papers to identify additional papers [65].”
The Circular Economy – A new sustainability paradigm? (2017, background): “This review started with a sample of relevant papers published by highly cited journals and academics, which was then followed by a semi-structured snowballing approach (Wohlin, 2014), to capture both established and emerging conceptual trends (see Figure 5).”
A Systematic Review of Digital Behaviour Change Interventions for More Sustainable Food Consumption (2019, positive): “After removing duplicates and applying the inclusion and exclusion criteria, we used backward and forward snowballing to (1) search through the references of all selected papers (backward snowballing) and (2) search for more recent papers that referred to any of the selected papers to find relevant ”
Learning from Failure and Success: The Challenges for Circular Economy Implementation in SMEs in an Emerging Economy (2021, methodology): “Finally, a snowball procedure was carried out, in which additional papers were identified using the reference lists [39].”
The AI-Based Cyber Threat Landscape (2020, methodology): “Then, we complemented the manual search with backward and forward snowballing technique of the papers that belonged to the three defined categories for a better coverage of related studies, using the snowballing procedure guidelines [84].”
Students’ Misconceptions and Other Difficulties in Introductory Programming (2017, methodology): “The first step of the snowballing approach is to identify a starter set of papers regarding a topic (Wohlin 2014).”
A systematic mapping study on technical debt and its management (2015, positive): “Second, we employed the “snowballing” technique (Wohlin, 2014) to include the potential studies in the references of the selected studies retrieved from the database searches.”
Software Testing With Large Language Models: Survey, Landscape, and Vision (2023, positive): “Next, to mitigate the risk of omitting relevant literature from this survey, we also perform backward snowballing [130] by inspecting the references cited by the collected papers so far.”
Cryptocurrency trading: a comprehensive survey (2020, methodology): “To ensure a high coverage, we adopted the so-called snowballing [222] method on each paper found through these keywords.”
23 Ways to Nudge: A Review of Technology-Mediated Nudging in Human-Computer Interaction (2019, background): “From the 71 articles, 14 papers were added following Wohlin guidelines for snowballing in systematic reviews [97].”
A Survey of Imitation Learning Methods, Environments and Metrics (2024, methodology): “In this work, we use a snowballing approach [115] and two different surveys: Hussein et al. [47] from 2017 and Zheng et al. [121] from 2021 as a starting point.”
A review on the contribution of crop diversification to Sustainable Development Goal 1 “No poverty” in different world regions (2019, background): “It refers to using the reference list of an article or the citations to the article to identify additional articles (Wohlin, 2014).”
A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence (2021, methodology): “The present survey has been undertaken as a systematic literature review following the guidelines by Kitchenham and Charters [58], Kitchenham et al. [59], and Wohlin [60].”