Is Bayesian Analysis Ready for Use in Phase III Randomized Clinical Trials?

Bayesian analysis is attractive (primarily) because it uses available data from other studies to potentially reduce study sample size.¹ Bayesian approaches begin by mathematically assuming a “prior distribution” for the treatment effect, then adjusts this distribution using the results of the Phase III study to produce a “posterior distribution.” With even modest information available, this approach will result in substantial improvements in efficacy and reduced sample size. On the surface, ignoring information seems at the least foolish, and at the most unethical.

There are numerous technical aspects of trial design (such as “adaptive trial” methods) for study monitoring available using the Bayesian approach.² However, alternatives to these approaches are available in the frequentist approach.^3–6 We consider these issues to be tangential to the primary “reduction in sample size” argument for using the Bayesian approach.

Although Bayesian approaches have become more acceptable for statistical analyses in general over the past few decades, the majority of Phase III trials are conducted under the frequentist approach. We suggest that this is because of rather fundamental issues associated with Bayesian Phase III trials. So why would we not use available information to reduce sample size?

We Have a Difficult Time Agreeing What We Know

The choice of studies guiding the “prior” and the relative value to be assigned to each study is largely subjective. The weight chosen is often a function of statistical precision, which provides more weight to a large poorly conducted trial than a smaller well-conducted study. It is not clear whether to include studies conducted in “almost” the same population, nor is it clear whether to include information from epidemiological studies.

Given the same Phase III trial, the use of different “priors” will result in different “posteriors.” Unless there is agreement on what is “known” before the study, this introduces the possibility that different interpretations could be drawn because of disagreements of what was “known.” This could be especially problematic when the study sponsor has a financial stake in the outcome of the study since critics could argue that the previous information was stacked to favor the desired finding. Although there are many disagreements on the interpretation of frequentist trials, it is not inherent from the design.

We Have To Be Careful To Know Exactly the Correct Amount

With little or no previous information available, there is no meaningful reduction in sample size using Bayesian methods. The Bayesian approach becomes increasingly attractive with increasing previous information.

However, substantial a priori knowledge introduces potential ethical concerns in the conduct of the trial. Although investigators may have their individual priors, equipoise is reflected in a societal belief of equal likelihood that either treatment is superior. It is not clear that equipoise exists if the study “officially” assumes a more likely winner. Such an assumption may imply there is not true informed consent. However, substantial recruitment barriers are likely if true informed consent is provided (ie, consider the impact of a statement that “we are only 70% sure the new treatment is superior and therefore would like to randomize you to potentially receive either treatment”). This is not an issue in frequentist trials in which the trial position of equipoise is present.

We May Know Something That Is (Systematically) Incorrect

There are 2 reasons why previous studies may provide incorrect information—sampling variation and bias. Bayesian approaches appropriately adjust for sampling variation, but biases are “carried forward” to affect the posterior distributions. There is no study absolutely unbiased; hence, we are not discussing the presence but rather the magnitude of “carry-forward” bias.

We suggest that even under the theoretical assumption that previous studies are conducted bias-free, the use of Bayesian approaches may introduce bias through a process akin to the file drawer problem in meta-analysis. Because of their cost and complexity, Phase III trials are not proposed without strong supportive evidence from earlier studies. Suppose a number of studies using potential drugs are evaluated. Among those studies, some will suggest efficacy and will be “encouraged” for Phase III trials, whereas others not showing efficacy are “discouraged.” Because of this differential selection process, the estimated effect of studies leading to Phase III trials is positively biased. These biased estimates serve as the foundation for the “prior” for the Phase III trial and subsequently result in biased estimates of treatment effect. This bias is introduced by the transition from Phase II to Phase III studies using Bayesian approaches but is avoided by the independent replication of the frequentist approach.

Conclusions

It is a difficult decision to ignore available information when making a decision, and as such the underlying goal of Bayesian analysis is laudable (seductive?). However, should trials not be an independent confirmation of our previous work? We suggest that answering the siren’s song of Bayesian analysis for Phase III trials introduces additional issues in study interpretation, ethical issues, and bias (some researchers seem to have better hearing than others).

Footnotes