An Examination of the Sufficiency of Small Qualitative Samples

By D. Young, E. Casey. Social work research (Print), 2018

189 citations recorded by the source corpus.

Qualitative researchers often confront dilemmas regarding determining what constitutes a robust sample size. The challenge is to find a sample that will produce thorough and meaningful findings while minimizing burden on participants and expenditure of scarce resources. Retrospectively considering three distinct qualitative research studies inclusive of individual interviewing and focus group data collection, minimum sample sizes needed to adequately include the themes and codes in each area of inquiry were examined through multiple random draws of smaller sub-samples. Research questions included: 1) What minimum sample size is needed to adequately identify codes within the data? 2) What minimum sample size is needed to ensure that all larger themes are partially represented? 3) What minimum sample size is needed to fully realize the complete dimensionality of all themes? 4) Are minimum sample sizes needed consistent across different substantive areas of exploration and modes of data collection? Sufficiency of Small Samples 2 Significant coverage of codes ranged from a minimum sample size of 6-9, partial theme representation required minimum sample sizes of 4-6, and substantial theme completion necessitated sample sizes of 7-10 cases across the projects. Additional cases added nuance to identified themes, but the vast majority of codes and themes were present in small samples. Sufficiency of Small Samples 3 Qualitative researchers often must make decisions about anticipated sample sizes in advance of data collection. Estimates are typically required for human subjects review committees, grant applications, and resource planning purposes. Once a study is underway or completed, researchers must evaluate whether the sample has been robust enough to address the research aims. The challenge is to find a sample that will produce thorough and meaningful findings while minimizing unnecessary burden on participants and expenditure of scarce resources such as time and research dollars. Currently, little guidance is available regarding what minimum sample size is needed to adequately identify the themes and codes in an area of inquiry. Additionally, sample sizes needed to reach theme and code saturation across different qualitative methodologies or data analysis approaches is under-studied. Although researchers often cite having achieved saturation as a reason to conclude sampling, details regarding how saturation was determined are not provided for the most part (Bowen, 2008; Francis et al., 2010). Gentles and colleagues (2015) conducted an overview of the literature from influential authors within the traditions of grounded theory, phenomenology, and case study. They noted the lack of clarity relative to sample size and saturation. Guetterman (2015) looked at the most-cited empirical articles in the fields of Education and Health Sciences from 2008 through 2012 within 5 qualitative research approaches to assess specific samples sizes and the rationale for sample sizes. Sample size across the 51 studies varied widely, and most articles did not include a discussion of saturation or the adequacy of the sample. In an effort to provide empirically-based guidance about appropriate minimum sample sizes for qualitative studies, researchers have recently begun to conduct methodological studies that examine the point at which data saturation occurs. Guest, Bunce, and Johnson (2006) operationalized data saturation “as the point in data collection and analysis when new Sufficiency of Small Samples 4 information produces little or no change to the codebook” (p. 65). They reviewed transcripts from a previous study in sets of 6, according to the order in which individual interviews had been conducted at two research sites. They noted theme and code development, asking whether 6 interviews yielded as much data as 12, 18, or 24, etc., interviews. They found that 73% of codes were identified in the first 6 interviews and 92% within the first 12 interviews. Examining this same question with focus group data, as opposed to individual interviews as in the earlier study, Guest, Namey, and McKenna (2016) found that 60% of their 94 codes were found in the first focus group, 84% in the first 3 groups, and 90% by 6. When the focus groups were randomly ordered to assess for temporal bias, the results remained consistent. Other researchers have examined the question of minimum sample size using different definitions of data saturation, sometimes referred to as code saturation. Extending Guest et al.’s (2006) findings to cross-cultural research, Hagaman and Wutich (2016) considered 3 repetitions of a theme by different interviewees as identification of that theme, and found that 16 interviews were enough to identify themes from homogeneous groups with 20-40 needed to identify metathemes that cut across cultures and study sites. Francis and colleagues (2010) considered data saturation to be achieved when no new ideas emerged with 3 additional interviews. Using this strict stopping criterion of 3, they examined interview data from two different studies and found that saturation was achieved in one study at 17 interviews, with no new data emerging after the 14 interview, and was not yet determined in the 14 interviews available to them from the second study. Even so, the majority of themes (92% and 86% in study one and two, respectively) emerged in the first 6 interviews. Finally, Hennink and colleagues (2016) proposed that code saturation is the point at which you have “heard it all,” but that meaning saturation is the point when you “understand it all” (p. 15). In their study on patient retention with 25 Sufficiency of Small Samples 5 individuals, they found that 84% of codes were identified by the 6 interview and 91% by the 9 interview. It took 16-24 interviews, however, to understand all the dimensions of 9 central codes, achieving meaning saturation. These findings suggest that under some study conditions, rich qualitative findings can be discovered with relatively small sample sizes. Further determining the parameters under which this applies would be helpful to researchers and research participants alike. Most efforts thus far have been done with studies utilizing individual interviews and many, although not all are within the medical field. Additionally, examinations of minimal required sample sizes that examine available interviews once, in the order they were collected, raise concerns about possible temporal bias. We sought to examine the minimum sample sizes needed to adequately include the themes and codes in areas of inquiry within the field of social work. Considering three distinct qualitative research studies inclusive of both individual interviewing and focus group data collection approaches, we address the following research questions: 1) What minimum sample size is needed to adequately identify codes (smaller units of meaning) within the data? 2) What minimum sample size is needed to ensure that all larger themes are partially represented by at least one of the codes that comprises that theme? 3) What minimum sample size is needed to fully realize the complete dimensionality of all themes by including all assigned codes? 4) Are minimum sample sizes needed consistent across different substantive areas of exploration and different modes of data collection, specifically individual interviews and focus groups? To address temporal bias, we address these questions by examining multiple random draws of various sample sizes within each included qualitative study.

Subfields: Education

View Paper: https://www.semanticscholar.org/paper/ba1b961a9e7a4cfcdfb228e9fd7a95be12d41a7c