Stopping a trial early – and then what?
Abstract
Background
This article addresses a problem arising when a trial shows such strong evidence of benefit of the tested intervention that it stops early with an observed effect size for the experimental treatment that is statistically significantly better than the control. Within the classical frequentist framework of group sequential trials, the observed estimated effect size, the associated naïve confidence interval, and the p-value are all biased estimates of the true values. The bias is in the direction of the overestimation of the treatment effect, creation of narrower confidence intervals than appropriate, and a p-value that is too small.
Purpose
To discuss methods for correcting the bias in observed effect sizes, confidence intervals, and p-values for trials stopped early and to show the extent to which such correction would have modified the conclusions of the Randomized Aldactone Evaluation Study (RALES).
Results
In RALES, the effect of not correcting for bias is negligible.
Limitations
This article does not show general results; it only explores a few examples that use conservative methods for early stopping. It does not consider sequential methods that allow a relatively high probability of stopping early.
Conclusions
This article points out that there is no unique solution to the correction of the p-value, but it recommends stagewise ordering, which states that earlier stopping of a trial is ipso facto stronger evidence of effect than later stopping so long as the stopping is governed by a monitoring boundary that preserves the Type I error rate. Associated with stagewise ordering is a method for calculating the estimated effect size and its confidence interval. In the RALES trial, which stopped at 50% information time, the corrections to the estimated values are small.
Get full access to this article
View all access and purchase options for this article.
References
1. O’Brien PC, Fleming TR. A multiple testing procedure for clinical trials. Biometrics 1979; 35: 549–56.
2. Lan K, DeMets D. Discrete sequential boundaries for clinical trials. Biometrika 1983; 70: 659–63.
3. Pinheiro JC, DeMets DL. Estimating and reducing bias in group sequential designs with Gaussian independent structure. Biometrika 1997; 84: 831–43.
4. Qu RP, DeMets DL. Bias correction in group sequential analysis with correlated data. Stat Sin 1999; 9: 939–52.
5. Li Z, DeMets DL. On the bias of estimation of a Brownian motion drift following group sequential tests. Stat Sin 1999; 9: 923–37.
6. Lee JW, DeMets DL. Estimation following group sequential tests with repeated measurements data. Comput Stat Data Anal 1999; 32: 69–77.
7. Yusuf S, Zhao F, Mehta SR, et al.; The Clopidogrel in Unstable Angina to Prevent Recurrent Events Trial Investigators. Effects of clopidogrel in addition to aspirin in patients with acute coronary syndromes without ST-segment elevation. N Engl J Med 2001; 345: 494–502.
8. Pitt B, Zannad F, Remme WJ, et al. The effect of spironolactone on morbidity and mortality in patients with severe heart failure. N Engl J Med 1999; 341: 709–17.
9. MERIT-HF Study Group. Effect of metoprolol CR/XL in chronic heart failure: Metoprolol CR/XL Randomized Intervention Trial in Congestive Heart Failure (MERIT-HF). Lancet 1999; 353: 2001–07.
10. Wedel H, DeMets D, Deedwania P, et al. Challenges of subgroup analyses in multinational clinical trials: Experiences from the MERIT-HF trial. American Heart Journal 2001; 142: 502–11.
11. Motzer RJ, Hutson TE, Tomczak P, et al. Sunitinib versus interferon alfa in metastatic renal-cell carcinoma. N Engl J Med 2007; 356: 115–24.
12. Bassler D, Briel M, Montori VM, et al. Stopping randomized trials early for benefit and estimation of treatment effects: Systematic review and meta-regression analysis. JAMA 2010; 303: 1180–87.
13. Berry SM, Carlin BP, Connor J. Letter: Bias and trials stopped early for benefit. JAMA 2010; 304: 156.
14. Goodman S, Berry D, Wittes J. Letter: Bias and trials stopped early for benefit. JAMA 2010; 304: 157.
15. Korn EL, Freidlin B, Mooney M. Letter: Bias and trials stopped early for benefit. JAMA 2010; 304: 157–58.
16. Ellenberg SS, DeMets DL, Fleming TR. Letter: Bias and trials stopped early for benefit. JAMA 2010; 304: 158–59.
17. Proschan M, Lan K, Wittes J. Statistical Monitoring of Clinical Trials: A Unified Approach. Springer, New York, 2006.
18. Lan K, Wittes J. The B-value: A tool for monitoring data. Biometrics 1988; 44: 579–85.
19. Emerson SS, Fleming TR. Parameter estimation following sequential hypothesis testing. Biometrika 1990; 77: 875–92.
20. Liu A, Hall WJ. Unbiased estimation following a group sequential test. Biometrika 1999; 86: 71–78.
21. Kim K, DeMets DL. Confidence intervals following group sequential tests in clinical trials. Biometrics 1987; 43: 857–64.
22. Kim K. Point estimation following group sequential tests. Biometrics 1989; 45: 613–17.
23. Reboussin DM, DeMets DL, Kim K, et al. Programs for computing group sequential boundaries using the Lan-DeMets method, v. 2.1, 2003. Available at: http://www.biostat.wisc.edu/landemets
24. Wittes J, Palensky J, Asner D, et al. Experience collecting interim data on mortality: An example from the RALES study. Curr Control Trials Cardiovasc Med 2001; 2: 59–62.
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: August 9, 2012
Issue published: December 2012
PubMed: 22879573
Authors
Metrics and citations
Metrics
Article usage*
Total views and downloads: 226
*Article usage tracking started in December 2016
Altmetric
See the impact this article is making through the number of times it’s been read, and the Altmetric Score.
Learn more about the Altmetric Scores
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 11 view articles Opens in new tab
Crossref: 12
-
Point estimation for adaptive trial designs II ...
-
Point estimation for adaptive trial designs I: A methodological review
-
The adaptive designs CONSORT extension (ACE) statement: a checklist wi...
-
Multigroup, Adaptively Randomized Trials Are Advantageous for Comparin...
-
The Adaptive designs CONSORT Extension (ACE) statement: a checklist wi...
-
Adaptive trial designs in diagnostic accuracy research
-
Randomised trials with provision for early stopping for benefit (or ha...
-
Short-course regimens of rifapentine plus isoniazid to treat latent tu...
-
Quantifying the bias in the estimated treatment effect in randomized t...
-
Stopping guidelines for an effectiveness trial: what should the protoc...
-
An Investigation of the Shortcomings of the CONSORT 2010 Statement for...
-
An Interview with Janet Wittes, President of Statistics Collaborative
Figures and tables
Figures & Media
Tables
View Options
Get access
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
loading institutional access options
SCT members can access this journal content using society membership credentials.
SCT members can access this journal content using society membership credentials.
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
