Quality in qualitative evaluation: A framework for assessing research evidence. A quality framework.

By L. Spencer, J. Ritchie, Jane Lewis, L. Dillon. 2003

858 citations recorded by the source corpus.

This document presents a framework for appraising the quality of qualitative evaluations. It was developed with particular reference to evaluations concerned with the development and implementation of social policy programmes and practice. The framework was devised as part of a programme of research conducted on behalf of the Cabinet Office.The research on which the framework was built involved: a comprehensive review of the literature on qualitative research methods relating to standards in qualitative research; a review of qualitative research methods used in Government funded evaluation studies; a review of existing frameworks for assessing quality in qualitative research; exploratory interviews with a range of people who have an interest in quality assessment of qualitative research and/or policy-related evaluations. These included academics who have written about qualitative research from either a theoretical or empirical perspective; authors of existing frameworks; research practitioners; commissioners and funders; and policy-makers who have used qualitative research evidence in the development and evaluation of policies; a workshop involving the above groups to refine the framework initially developed; a trial application of the framework to a small number of studies. All these strands of activity have heavily influenced the content of the framework and the premises that surround its operation.

View Paper: https://www.semanticscholar.org/paper/29cb97c154235c26dd5a42017d0f65e7d8b524d5

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Daniela Soares Cruzes
2. Francesca Bastagli
3. Neal R Haddaway
4. Drishti Yadav
5. J. Hagen‐Zanker
6. M. Dixon-Woods
7. P. Twining
8. Luke Harman

Citations in the corpus

• Improving response rates using a monetary incentive for patient completion of questionnaires: an observational study (2007, unknown): “Assessing the quality of qualitative research has attracted much debate and there is little consensus regarding how quality should be assessed, who should assess quality, and, indeed, whether quality can or should be assessed in relation to 'qualitative' research at all [43, 22, 44, 45].”
• Trade unions and climate change : The jobs versus environment dilemma (2011, methodology): “Qualitative data are collected selectively according to the research question (Lincoln and Guba, 1985; Styles, 1993; Spencer et al., 2003; Yardley, 2000) Yardley (2000) suggests that conducting and analysing qualitative research should be based on four broad principles: sensitivity to context;…”
• Making literature reviews more reliable through application of lessons from systematic reviews (2015, methodology): “For qualitative data, a variety of critical appraisal tools have been produced that can be adapted for use in conservation (Spencer et al. 2003; DfID 2014).”
• Talking Science: The research evidence on the use of small group discussions in science teaching (2010, unknown): “As much of the work on small group discussion draws extensively on qualitative approaches, the work reported here extended the EPPI review methods to draw on the guidance and framework for assessing research evidence in qualitative research studies (Spencer, Ritchie, Lewis, & Dillon, 2003 ).”
• Research synthesis in software engineering: A tertiary study (2011, methodology): “[54] This framework was developed by the UK Cabinet Office to support work in Departments.”
• Criteria for Good Qualitative Research: A Comprehensive Review (2021, background): “Among qualitative researchers, the dilemma of governing the measures to assess the quality of research is not a new phenomenon, especially when the virtuous triad of objectivity, reliability, and validity (Spencer et al., 2004) are not adequate.”
• The Impact of Economic Resource Transfers to Women Versus Men (2012, methodology): “The framework is based on the 18 questions for appraising qualitative studies in Spencer et al. (2003), adapted to include elements from the 10 questions for appraising qualitative studies used in the Critical Appraisal Skills Programme (2006).”
• The impact of land property rights interventions on investment and agricultural productivity in developing countries: a systematic review (2017, methodology): “For the qualitative synthesis, we accepted studies with clearly defined research objectives and links to relevant literature, and that provided details on context, sample selection and data collection methods (Kuper, Lingard, and Levinson 2008; Spencer et al. 2003; Waddington et al. 2010).”
• Analysing Qualitative Longitudinal Research in Evaluations (2007, unknown): “The role of qualitative methods in evaluative research is now well established, and it is seen as having a number of important contributions to make (Finch, 1986; Shaw, 1999; Rist, 2000; Patton, 2002; see Spencer et al., 2003 for a brief overview).”