Systems for grading the quality of evidence and the strength of recommendations I: Critical appraisal of existing approaches The GRADE Working Group

By D. Atkins, M. Eccles, S. Flottorp, G. Guyatt, D. Henry, S. Hill, A. Liberati, Dianne O'Connell, Andrew D. Oxman, B. Phillips and 4 more. BMC Health Services Research, 2004

1,198 citations recorded by the source corpus.

BackgroundA number of approaches have been used to grade levels of evidence and the strength of recommendations. The use of many different approaches detracts from one of the main reasons for having explicit approaches: to concisely characterise and communicate this information so that it can easily be understood and thereby help people make well-informed decisions. Our objective was to critically appraise six prominent systems for grading levels of evidence and the strength of recommendations as a basis for agreeing on characteristics of a common, sensible approach to grading levels of evidence and the strength of recommendations.MethodsSix prominent systems for grading levels of evidence and strength of recommendations were selected and someone familiar with each system prepared a description of each of these. Twelve assessors independently evaluated each system based on twelve criteria to assess the sensibility of the different approaches. Systems used by 51 organisations were compared with these six approaches.ResultsThere was poor agreement about the sensibility of the six systems. Only one of the systems was suitable for all four types of questions we considered (effectiveness, harm, diagnosis and prognosis). None of the systems was considered usable for all of the target groups we considered (professionals, patients and policy makers). The raters found low reproducibility of judgements made using all six systems. Systems used by 51 organisations that sponsor clinical practice guidelines included a number of minor variations of the six systems that we critically appraised.ConclusionsAll of the currently used approaches to grading levels of evidence and the strength of recommendations have important shortcomings.

Subfields: General Health Professions, Public Health, Environmental and Occupational Health, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/ed2f8468159e11cd99f395bc5ac35fb630d56af5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In General Health Professions

In Public Health, Environmental and Occupational Health

In Statistics and Probability

Citations in the corpus

Empirically Supported Treatment: Recommendations for a New Model (2015, methodology): “…and (b) a committee-based appraisal of the evidence, using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system (Atkins et al., 2004; Guyatt et al., 2006, 2008) to assess the quality of relevant evidence and degree to which beneﬁts are established in excess of…”
American Society of Hematology 2019 guidelines for immune thrombocytopenia. (2019, neutral): “The certainty was categorized into 4 levels ranging from very low to high.(6-8)”
Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework (2017, neutral): “(two reviews: 6%); Grade scale [42] (one review: 3%),”
Practice parameters for the use of actigraphy in the assessment of sleep and sleep disorders: an update for 2007. (2007, methodology): “2 (4-12) Significant differences in number of movements according to diagnostic category 54% diagnosed with sleep disordered breathing, 38% primary snoring or normal PSG, and 15% with PLMD Actiwatch-64 (MM) Start: End: Durat:overnight Actiwatch-PLMs software Derived from ASDA criteria, w/ 2 sec asse”
Endoscopic submucosal dissection for superficial gastrointestinal lesions: European Society of Gastrointestinal Endoscopy (ESGE) Guideline - Update 2022. (2022, background): “All selected important articles were individually assessed and graded by the level of evidence and strength of recommendation, according to the GRADE system [4, 5].”
American Society of Hematology 2018 guidelines for management of venous thromboembolism: prophylaxis for hospitalized and nonhospitalized medical patients. (2018, methodology): “The certainty was categorized into 4 levels ranging from very low to high.(14-16)”
Management of epithelial precancerous conditions and lesions in the stomach (MAPS II): European Society of Gastrointestinal Endoscopy (ESGE), European Helicobacter and Microbiota Study Group (EHMSG), European Society of Pathology (ESP), and Sociedade Portuguesa de Endoscopia Digestiva (SPED) guideli (2019, methodology): “Each working group rated the quality level of the available evidence and the strength of recommendations using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) process [8, 9].”
Mindfulness-based interventions for psychiatric disorders: A systematic review and meta-analysis. (2018, methodology): “Considerations for minimizing bias in individual studies were drawn from both Jadad's criteria as well as the GRADE system (Atkins et al., 2004; Jadad et al., 1996).”
Prevalence of depression in survivors of acute myocardial infarction (2007, methodology): “The evidence-grading scheme followed the approach recommended by the International GRADE Working Group, and reflected the quantity, quality, and consistency of the evidence.(24) Assessment of the methodological quality of each study was performed independently by each reviewer and reflected the stud”