Systematic literature reviews in software engineering - A systematic literature review

By Barbara A. Kitchenham, Pearl Brereton, David Budgen, Mark Turner, John Bailey, Stephen G. Linkman. Information and Software Technology, 2009

4,399 citations recorded by the source corpus.

Background: In 2004 the concept of evidence-based software engineering (EBSE) was introduced at the ICSE04 conference. Aims: This study assesses the impact of systematic literature reviews (SLRs) which are the recommended EBSE method for aggregating evidence. Method: We used the standard systematic literature review method employing a manual search of 10 journals and 4 conference proceedings. Results: Of 20 relevant studies, eight addressed research trends rather than technique evaluation. Seven SLRs addressed cost estimation. The quality of SLRs was fair with only three scoring less than 2 out of 4. Conclusions: Currently, the topic areas covered by SLRs are limited. European researchers, particularly those at the Simula Laboratory appear to be the leading exponents of systematic literature reviews. The series of cost estimation SLRs demonstrate the potential value of EBSE for synthesising evidence and making it available to practitioners.

Subfields: Artificial Intelligence, Information Systems

View Paper: https://www.semanticscholar.org/paper/a963d05b9d4acd347ad528e7d098eb53d8f555a2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

Citations in the corpus

A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence (2021, methodology): “The present survey has been undertaken as a systematic literature review following the guidelines by Kitchenham and Charters [58], Kitchenham et al. [59], and Wohlin [60].”
Guidelines for conducting systematic mapping studies in software engineering: An update (2015, methodology): “A few mapping studies included in our map have used manual search as their only study identification approach [60,30].”
Microservices: A Systematic Mapping Study (2016, background): “SLRs and SMS identify, classify and synthesize a comparative overview of state-of-the-research and enable (Petersen et al., 2008; Kitchenham et al., 2009).”
Machine learning and deep learning approach for medical image analysis: diagnosis to detection (2022, methodology): “The guidelines given in [49, 50]”
Machine Learning Operations (MLOps): Overview, Definition, and Architecture (2022, methodology): “As a first step, we conduct a structured literature review [20,43] to obtain an overview of relevant research.”
Evaluating prediction systems in software project estimation (2012, background): “However, we still tend to find inconclusive results from systematic reviews (or meta-analyses) [19].”
A systematic review of systematic review process research in software engineering (2013, background): “Since then, systematic reviews (SRs) have become increasingly popular in empirical software engineering as demonstrated by three tertiary studies reporting the numbers of such studies [15,12,4].”
Systematic literature reviews in software engineering - A tertiary study (2010, methodology): “These are the same criteria that were used to evaluate quality in the original tertiary study 1 [12], except for scoring N for Q3 if papers collected quality data but did not use it.”
Multimedia tools in the teaching and learning processes: A systematic review (2020, background): “In order to ensure appropriate primary search terms are used and relevant papers are carefully selected for the review purpose, the literature search method of Kitchenham et al. (2009) was adopted.”
Towards Sustainable Energy: A Systematic Review of Renewable Energy Sources, Technologies, and Public Opinions (2019, background): “A SLR is a means of identifying, evaluating and interpreting available research relevant to a particular research question or topic area [34].”
Software development in startup companies: A systematic mapping study (2014, background): “A wide body of knowledge has been created in recent years through several empirical studies, investigating how companies leverage software engineering (SE) [1, 2].”
Experimentation in Software Engineering (2012, background): “report 53 unique systematic literature reviews in software engineering being published between 2004 and 2008 [103,104].”
A systematic review on regression test selection techniques (2010, background): “recently published a review of 20 systematic reviews in software engineering 2004-2007[28].”
A Metaverse: Taxonomy, Components, Applications, and Open Challenges (2022, background): “reviews (SLRs) techniques to obtain reliable references [11].”
A survey of the use of crowdsourcing in software engineering (2017, methodology): “The process we adopted was borrowed from the conventions used for systematic literature reviews [16], [17].”
Does the technology acceptance model predict actual use? A systematic literature review (2010, background): “is changing however, and a recent review of SLRs within software engineering found that the number of SLRs had increased from six in 2004 to 15 in 2007 [33].”