Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study
By Neil Shephard, Amy L Whitehead. Trials, 2014
634 citations recorded by the source corpus.
BackgroundExternal pilot or feasibility studies can be used to estimate key unknown parameters to inform the design of the definitive randomised controlled trial (RCT). However, there is little consensus on how large pilot studies need to be, and some suggest inflating estimates to adjust for the lack of precision when planning the definitive RCT.MethodsWe use a simulation approach to illustrate the sampling distribution of the standard deviation for continuous outcomes and the event rate for binary outcomes. We present the impact of increasing the pilot sample size on the precision and bias of these estimates, and predicted power under three realistic scenarios. We also illustrate the consequences of using a confidence interval argument to inflate estimates so the required power is achieved with a pre-specified level of confidence. We limit our attention to external pilot and feasibility studies prior to a two-parallel-balanced-group superiority RCT.ResultsFor normally distributed outcomes, the relative gain in precision of the pooled standard deviation (SDp) is less than 10% (for each five subjects added per group) once the total sample size is 70. For true proportions between 0.1 and 0.5, we find the gain in precision for each five subjects added to the pilot sample is less than 5% once the sample size is 60. Adjusting the required sample sizes for the imprecision in the pilot study estimates can result in excessively large definitive RCTs and also requires a pilot sample size of 60 to 90 for the true effect sizes considered here.ConclusionsWe recommend that an external pilot study has at least 70 measured subjects (35 per group) when estimating the SDp for a continuous outcome. If the event rate in an intervention group needs to be estimated by the pilot then a total of 60 to 100 subjects is required. Hence if the primary outcome is binary a total of at least 120 subjects (60 in each group) may be required in the pilot trial. It is very much more efficient to use a larger pilot study, than to guard against the lack of precision by using inflated estimates.
Subfields: Statistics and Probability
View Paper: https://www.semanticscholar.org/paper/4b2fbafae462d72b25c4366c5bd529eeaefae10c
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Statistics and Probability
- Amy L Whitehead
- Daniël Lakens
- Michael Campbell
- C. Cooper
- S. Julious
- Farid Anvari
- C. Brydges
- Vito M. R. Muggeo
Citations in the corpus
- CONSORT 2010 statement: extension to randomised pilot and feasibility trials (2016, neutral): “There are a number of relevant papers.[54-56] Amongst these, a recent paper by Whitehead et al suggests that size of a pilot trial should be related to the size of the future definitive trial.”
- Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable (2015, background): “Julious(16) 24 Kieser and Wassmer(12) 20–40 Browne(10) 30 Sim and Lewis(11) 55 Teare et al.(17) 70 Whitehead et al.”
- Sample Size Justification (2021, positive): “It is simply not possible to perform an accurate power analysis on the basis of an effect size estimate from a study that might be biased and/or had a small sample size (Teare et al., 2014).”