Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist

By C. Terwee, L. Mokkink, Dirk L. Knol, R. Ostelo, Lex M. Bouter, HC de Vet. Quality of Life Research, 2011

1,810 citations recorded by the source corpus.

BackgroundThe COSMIN checklist is a standardized tool for assessing the methodological quality of studies on measurement properties. It contains 9 boxes, each dealing with one measurement property, with 5–18 items per box about design aspects and statistical methods. Our aim was to develop a scoring system for the COSMIN checklist to calculate quality scores per measurement property when using the checklist in systematic reviews of measurement properties.MethodsThe scoring system was developed based on discussions among experts and testing of the scoring system on 46 articles from a systematic review. Four response options were defined for each COSMIN item (excellent, good, fair, and poor). A quality score per measurement property is obtained by taking the lowest rating of any item in a box (“worst score counts”).ResultsSpecific criteria for excellent, good, fair, and poor quality for each COSMIN item are described. In defining the criteria, the “worst score counts” algorithm was taken into consideration. This means that only fatal flaws were defined as poor quality. The scores of the 46 articles show how the scoring system can be used to provide an overview of the methodological quality of studies included in a systematic review of measurement properties.ConclusionsBased on experience in testing this scoring system on 46 articles, the COSMIN checklist with the proposed scoring system seems to be a useful tool for assessing the methodological quality of studies included in systematic reviews of measurement properties.

Subfields: Education, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/80858ac3671354dc3bf1b226c74eb8fe0513fb26

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Education

1. C. Terwee
2. L. Mokkink
3. C. Prinsen
4. Jordi Alonso
5. D.walaa badawy
6. HC de Vet
7. Lex M. Bouter

In Statistics and Probability

1. L. Mokkink
2. C. Terwee
3. C. Prinsen
4. HC de Vet
5. Lex M. Bouter
6. Jordi Alonso
7. D.walaa badawy

Citations in the corpus

• COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures (2017, methodology): “To determine risk of bias in a study on measurement properties, the methodological quality of a study is assessed using the ‘worst score counts’ principle [7].”
• Accuracy of Fitbit Devices: Systematic Review and Narrative Syntheses of Quantitative Data (2018, methodology): “We rated each dimension as excellent, good, fair, or poor quality based on a priori modifications to the COSMIN validation subscale scoring criteria appropriate for accuracy studies (Multimedia Appendix 4) [16].”
• COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study (2018, methodology): “A total rating per box can be obtained by taking the lowest rating of any item in the box (‘worst score counts’) [30].”
• A systematic review of evidence for the psychometric properties of the Strengths and Difficulties Questionnaire (2016, methodology): “It is possible that we identified a larger number of study limitations of studies than others have (Stone et al., 2010) as the COSMIN tool is relatively new (Mokkink et al., 2010; Terwee et al., 2012).”
• COSMIN guideline for systematic reviews of patient-reported outcome measures (2018, methodology): “In the absence of empirical evidence, the present COSMIN guideline for systematic reviews of PROMs is based on our experience that we (that is: the COSMIN steering committee) have gained over the past years in conducting systematic reviews of PROMs [11, 12], in supporting other systematic reviewers ”