Recommendations for data monitoring committees from the Clinical Trials Transformation Initiative
Abstract
Background/aims
Use of data monitoring committees to oversee clinical trials was first proposed nearly 50 years ago. Since then, data monitoring committee use in clinical trials has increased and evolved. Nonetheless, there are no well-defined criteria for determining the need for a data monitoring committee, and considerable variability exists in data monitoring committee composition and conduct. To understand and describe the role and function of data monitoring committees, and establish best practices for data monitoring committee trial oversight, the Clinical Trials Transformation Initiative—a public–private partnership to improve clinical trials—launched a multi-stakeholder project.
Methods
The data monitoring committee project team included 16 individuals charged with (1) clarifying the purpose of data monitoring committees, (2) identifying best practices for independent data monitoring committee conduct, (3) describing effective communication practices, and (4) developing strategies for training data monitoring committee members. Evidence gathering included a survey, a series of focus group discussions, and a 2-day expert meeting aimed at achieving consensus opinions that form the foundation of our data monitoring committee recommendations.
Results
We define the role of the data monitoring committee as an advisor to the research sponsor on whether to continue, modify, or terminate a trial based on periodic assessment of trial data. Data monitoring committees should remain independent from the sponsor and be composed of members with no relevant conflicts of interest. Representation on a data monitoring committee generally should include at least one clinician with expertise in the therapeutic area being studied, a biostatistician, and a designated chairperson who has experience with clinical trials and data monitoring. Data monitoring committee meetings are held periodically to evaluate the unmasked data from ongoing trials, but the content and conduct of meetings may vary depending on specific goals or topics for deliberation. To guide data monitoring committee conduct and communication plans, a charter consistent with the protocol’s research design and statistical analysis plan should be developed and agreed upon by the sponsor and the data monitoring committee prior to patient enrollment. We recommend concise and flexible charters that explain roles, responsibilities, operational issues, and how data monitoring committee recommendations are generated and communicated. The demand for data monitoring committee members appears to exceed the current pool of qualified individuals. To prepare a new generation of trained data monitoring committee members, we encourage a combination of didactic educational programs, practical experience, and skill development through apprenticeships and mentoring by experienced data monitoring committee members.
Conclusion
Our recommendations address data monitoring committee use, conduct, communication practices, and member preparation and training. Furthermore recommendations form the foundation for ongoing efforts to improve clinical trial oversight and enhance the safety and integrity of clinical research. These recommendations serve as a call to action for implementation of best practices that benefit study participants, study sponsors, and society.
Introduction
The use of data monitoring committees (DMCs) to oversee clinical trials has increased and evolved since the concept was introduced in 1967 by the Greenberg Report.1 Initial recommendations in that report were applied in National Institutes of Health (NIH)-sponsored cardiovascular trials to monitor trial conduct and safety and to recommend trial modifications or closure. Today, DMCs are occasionally used across therapeutic areas to oversee single trials, groups of trials, or entire portfolios of research related to an investigational intervention. Safeguarding clinical trial participants and monitoring interim safety and efficacy outcomes data in ongoing trials remain paramount responsibilities for DMCs, but variation in the structure and organization of DMCs exist. Membership and responsibilities of DMCs also may vary depending on the nature and goals of the trial.
The Clinical Trials Transformation Initiative, a public–private partnership whose mission is to develop and drive adoption of practices that will increase the quality and efficiency of clinical trials, initiated the DMC Project to address the identified issues in understanding the role, importance, and conduct of DMCs, and to recommend best practices for DMCs and for sponsors working with DMCs. The DMC Project Team included 16 representatives from a broad cross section of the clinical trials enterprise, including regulators, government and industry sponsors of clinical research, academics, contract research organizations, patient representatives, and clinical investigators. The project team developed recommendations for DMC use and conduct (Supplementary Appendix 1) based on their expertise and analysis of the findings from the project’s evidence-gathering activities. Our recommendations may apply to any DMC that is charged with monitoring an interventional trial regardless of sponsorship or funding source. While these recommendations focus on external DMCs (defined as an independent group of individuals, external to the sponsor, that conduct its activities outside of the sponsor organization), many of the principles and recommendations may also apply to internal DMCs that conduct similar activities within the sponsor organization.2
The objectives of the DMC Project were to (a) clarify the purpose of DMCs and the rationale for their use; (b) develop best-practice recommendations for the operation and optimal conduct of independent DMCs; (c) describe effective communication practices between independent DMCs and trial stakeholders (e.g. sponsors, investigators, and institutional review boards); and (d) identify strategies for preparing the next generation of DMC members.3
Methods
Approach
To address the objectives, the DMC Project Team employed three research strategies: a survey of 143 DMC members and organizers, a series of focus group discussions with 43 participants, and a 2-day expert meeting. Detailed methods and results of the survey and focus group discussions are described elsewhere.3
The expert meeting4 was conducted in July 2015 among 54 stakeholders representing academia, government agencies, industry, contract research organizations, patient representatives, and professional societies. Findings and key themes from the survey and focus group discussions were presented. The DMC Project Team used discussion from the meeting to refine recommendations through an iterative process based on consensus-building guidelines5 that focus on core values of inclusiveness, shared control, and flexibility.
Described herein are the primary outcomes of the DMC Project with emphasis on consensus-based, multi-stakeholder recommendations (Supplementary Appendix 1) for optimizing the operation and conduct of contemporary DMCs.
Results
Clarifying the role of the DMC
As use of DMCs has increased and evolved, confusion has emerged regarding the role of the DMC, which may contribute to unclear expectations between DMCs and other trial stakeholders. We sought to clarify the unique role of DMCs relative to roles of other groups involved in oversight of clinical trials.
The key difference between a DMC and other research oversight groups is that DMCs perform periodic benefit–risk assessments using available efficacy and safety outcomes data gathered during the course of a trial in order to provide the most optimal recommendations and advice to the sponsor and trial leadership. This necessitates close monitoring of the trial for “early definitive evidence of benefit, convincing evidence of harm, or sufficient evidence of no potential benefit to render continuation of the trial to be futile.”6 To adequately perform this important function, DMC members require full access to the unmasked safety and efficacy outcomes data during the course of the trial. The DMC must be able to review the accumulating data by treatment group to assess the benefit–risk balance for trial participants. We emphasize that interim analyses of unmasked trial data require thoughtful consideration and the utmost of care. Various statistical monitoring methods exist but were not discussed in this project and are beyond the scope of this article.
When reviewing trial data, bias must be minimized particularly in the assessment of study outcomes and attribution of adverse events. Therefore, independence from the trial sponsor is critical for the DMC to fulfill its central role of protecting vulnerable study participants from unpredictable harm that may arise during the course of a trial. Occasionally, this may require unscheduled meetings of the DMC and/or additional analyses without alerting the sponsor or study investigators.
Best practices for DMC conduct
Composition
The composition of a DMC must be carefully balanced to ensure effective monitoring of clinical trials. Representation on a DMC, at minimum, should include a clinician with expertise in the therapeutic area being studied and a biostatistician with expertise in statistical monitoring plans and analysis of clinical trial data. The designated chairperson—whether a clinician or statistician—must have experience with clinical trials and data monitoring. Other types of expertise (e.g. pharmacology, toxicology, and behavioral science) also may be required, and some trials by nature have challenging social, cultural, and ethical implications and may benefit from added expertise and diverse perspectives for effective evaluation and monitoring. In light of the increased complexity of clinical trials and interventions being evaluated, the inclusion of bioethicists and patient advocates should also be considered, particularly for trials evaluating high-risk interventions or involving vulnerable populations. Knowledge of research methodology and data analysis, and experience in clinical research are skills generally considered essential for any DMC member.
Selection of an effective DMC chairperson is critically important. The pivotal role of the DMC chair is not limited to trial monitoring, but extends to organizing the operational aspects of the committee and ensuring that DMC members have adequate resources and flexibility to do their work without hindrance or undue interference, particularly from sponsors and others with a vested interest in the trial outcome. Prior experience as a DMC member is essential for the chair. Importantly, the chair should be an accomplished leader and effective communicator who can skillfully manage meetings and create an environment that encourages cooperation and active participation of all DMC members. The chair should be capable of bringing consensus without being overbearing or forceful with personal conclusions or opinions. In addition, the DMC chair should have the necessary interpersonal skills to draw from the collective talents of all members in order to thoughtfully and effectively guide the process of monitoring and oversight.
Conflicts of interest
Prospective DMC members may have potential financial or intellectual conflicts of interest that could compromise their ability to objectively monitor a trial. Thus, conflict of interest must be regularly disclosed, assessed, and managed for all DMC members. At each meeting, members should be asked to declare any new conflicts, and report activities or connections with any parties that may introduce bias and influence their conduct. Activities or relationships deemed to have the potential to undermine independence of DMC members may result in disqualification from DMC service; therefore, both actual and perceived conflicts should be disclosed. Even the perception of a conflict of interest can damage the credibility of the DMC and raise questions about its conduct and recommendations.
Conversely, it is important to note that not all previous interactions with a sponsor are necessarily disqualifying. In some cases, identifying experts with highly specific skills and knowledge but without any connections to the study sponsor or investigators can be difficult. If concerns about conflicts of interest are taken to extremes, few qualified members would be available to serve on DMCs. Many minor conflicts that are unlikely to introduce bias (e.g. prior DMC service for the same sponsor for a different treatment intervention) can be addressed and managed by proper disclosures to the sponsor and other DMC members. However, some conflicts are so significant that they cannot be mitigated by the usual means and may require exclusion from DMC service for certain trials.
It should be emphasized that not all conflicts of interest are financial in nature. Scientists can have vested intellectual or research interests in the results of a given trial, which might impede their impartiality. Such conflicts must also be addressed on a case-by-case basis and may preclude service on a DMC.
Statistical Data Analysis Center
To support the DMC in fulfilling its role, a Statistical Data Analysis Center capable of preparing reports for or performing additional analyses that may be requested by the DMC is typically utilized. For the DMC to make optimal recommendations regarding the trial to the sponsor and trial leadership, planned interim analyses (based on the DMC Charter, trial protocol, and the statistical analysis plan) may necessitate unplanned analyses to provide insight regarding the interim safety and/or efficacy findings. Therefore, the Statistical Data Analysis Center should have access to all accumulating trial data beginning at trial initiation, possibly necessitating coordination between the Statistical Data Analysis Center and the trial’s data management group. It is not acceptable for the sponsor—either by requirement or by financial contract—to limit the scope of statistical work that is to be conducted by the Statistical Data Analysis Center. Instead, the Statistical Data Analysis Center contracts should allow for reasonable adjustments after trial initiation to ensure the sponsor does not unduly influence or restrict the type of work the Statistical Data Analysis Center conducts in support of the DMC. This approach would also minimize the chance that a sponsor is inadvertently informed about additional analyses requested by the DMC in the course of trial monitoring.
The Statistical Data Analysis Center should receive scheduled data transfers both prior to scheduled data reviews and during the period between reviews. Flexibility in the timing of these transfers is essential to aid the DMC in fulfilling its responsibilities. The tables, listings, and figures to be provided to the DMC during its meeting should be specified in advance and the templates approved by the DMC prior to its first data review. Changes to these templates may be requested during the trial, and there should be enough flexibility by the Statistical Data Analysis Center to implement these modifications.
DMC meetings
A best practice for DMC meetings is to hold an initial organizational meeting in order to orient and familiarize DMC members with their roles and responsibilities. All DMC meetings should be held at a neutral venue, avoiding sponsor offices or lavish accommodations. The inaugural meeting should ideally be held in person prior to the start of patient recruitment to allow DMC members to meet each other and review the DMC charter, protocol, and planned Statistical Data Analysis Center report templates. The protocol and statistical analysis plan should be readily available. The DMC members should have minimal sponsor interactions outside of the formal DMC meetings.
In addition to the DMC members, another key participant in the DMC meetings is the Statistical Data Analysis Center biostatistician. As the Statistical Data Analysis Center reports to and serves the DMC directly, the Statistical Data Analysis Center biostatistician should have an in-depth understanding of the data and how it is acquired, as well as comprehensive knowledge of the statistical analysis plan and protocol.
We recommend a face-to-face DMC meeting at least annually, but other meetings can be held via teleconference or web-based conferencing. Meetings can consist of open sessions (meetings in which individuals not directly involved in the DMC operations may attend) or closed sessions (meetings in which only DMC members and the Statistical Data Analysis Center statistician are permitted). Only blinded data are reviewed in open sessions. Regardless of trial sponsorship (i.e. commercial, government, or private foundation), review of unblinded data can only occur in the closed sessions without any representation or undue influence from the sponsor. Even during open sessions in which blinded data are reviewed and study progress is discussed, sponsor and trial leadership attendees generally should be limited to a few designated officials who are directly responsible for overseeing the trial for the sponsoring organization.
Effective communication practices
Charter
To inform DMC communication practices and address the overall oversight process, a charter that is carefully aligned with the research protocol and the statistical analysis plan should be developed by the sponsor in collaboration with the trial executive committee and with substantive input from the DMC. This important document should be agreed upon by the sponsor, executive committee, Statistical Data Analysis Center, and the DMC members prior to patient enrollment. After careful review of the charter, the protocol, and the statistical analysis plan, feedback from the DMC should be incorporated into the charter. The charter should clearly state the rationale for use of a DMC, broad goals, and the roles, responsibilities, and operational structure of the DMC relative to other clinical trial oversight groups. In addition, the charter should clearly describe the decision-making process of the DMC, describe how DMC recommendations are made, and include the following items: (1) composition, including the number and expertise areas of its members; (2) scheduled data transfers from the trial’s data management group to the Statistical Data Analysis Center; (3) the format (face-to-face, tele- or video-conference, open and closed session, etc.) and frequency (e.g. every 6 months) of meetings; and (4) the relationship and communication between DMC and Statistical Data Analysis Center, and other trial committees and stakeholders, including the trial sponsor.
The content of a DMC charter and the principles underlying it are not identical to those of the protocol and statistical analysis plan. By design, the latter documents are meant to be strictly followed, and any deviations need to be documented with substantive changes requiring amendments. In contrast, the DMC charter should be a succinct and user-friendly document that outlines a set of guiding principles for conduct of the DMC. While clearly aligned with the protocol and statistical analysis plan, the charter should avoid rigidity and legalism since it is not possible to anticipate and address all potential scenarios that could emerge during the course of an ongoing trial. Lengthy elements, such as table and figure templates to be included in DMC reports, should be relegated to the appendix section of the charter. Given the broad and flexible nature of the charter, amendments to this document should be infrequent. A critical aspect of the DMC charter is the monitoring guidelines for efficacy and safety outcomes.
DMC recommendations
The recommendation to continue, modify, or terminate a trial is the most important communication provided to the sponsor and trial leadership by the DMC. The DMC makes its recommendations based on benefit–risk assessments, and it is the sponsor who is ultimately responsible for acting upon these recommendations. Consensus should be sought among DMC members, and voting is generally discouraged. If differences of opinion persist, these are documented in the DMC minutes, and it is acceptable to describe these differences without attribution when issuing a statement or other formal communication.
As previously described, sponsors—and particularly the project team(s) directly involved in trial operations—often have a vested interest that may lead to a biased perspective on the research. Therefore, DMC trial recommendations and proposed modifications should be provided to a steering committee or sponsor leadership group authorized to act on these recommendations, and not to those directly involved with implementation of the trial.
The primary and preferred method of communicating the DMC’s recommendations to the sponsor is in written form. The DMC may also verbally brief the sponsor and/or trial leadership after the closed session, and the recommendations should be conveyed clearly and concisely.
When in agreement with the DMC’s recommendations, the sponsor should report these within an appropriate time period to institutional review boards and, in the case of trials performed under regulatory guidance, to the relevant regulatory authorities. Minor operational recommendations do not necessarily require regulatory reporting. Procedures for managing disagreements between the sponsor and the DMC should be described in the charter. Although consensus between the sponsor and DMC with respect to the recommendations is highly desirable, in case of an impasse, it is the sponsor’s decision whether to accept or reject the recommendations. The sponsor may choose to respond to the DMC through written comments, especially in the case of disagreement with the DMC’s recommendations. If the sponsor rejects the recommendations, this decision and its rationale should be reported promptly along with the written DMC recommendations to institutional review boards and to the appropriate regulatory agencies if the trial is under regulatory purview. Based on the information provided, the regulatory agencies and institutional review boards may reach their own independent conclusions and act accordingly within their respective authorities. At the end of the trial, all minutes and reports from the DMC meetings should be made available to the sponsor and trial leadership, as needed.
Preparing the next generation of DMC members
The pool of qualified individuals available to serve as DMC members may soon be inadequate to meet the current needs of the research enterprise, as demand for trained and qualified DMC members has risen and may continue to grow. In 2013, the Office of Inspector General at the US Department of Health and Human Services reported that the NIH faces challenges in the recruitment and training of DMC members. As a result, the Office of Inspector General7 recommended that NIH develop ways to recruit and train new DMC members. Although training is highly desirable prior to serving on a DMC, the vast majority of our survey respondents indicated that they had not received training and were unaware of DMC-specific training programs.3
The DMC Project also identified a growing need to prepare a new generation of qualified DMC members so that the pool of properly trained and experienced individuals does not dwindle. Preparing individuals to serve as DMC members is challenging because of the complexity of data monitored in clinical trials and the interpretation relative to the monitoring guidelines. Knowledge of research, familiarity with the study design, and unstructured on-the-job training are not sufficient to ensure that prospective DMC members are adequately qualified to serve on a DMC. While the skills needed for prospective DMC members are described in the literature, to date, nationally recognized training programs have not been established.
Effective training for DMC members should consist of a combination of didactic educational programs and practical experience. Didactic elements could include a review of the fundamentals of clinical trials, study design, data analysis, and the functions and responsibilities of DMCs. They should also focus on the aspects of DMC work that are different from the work conducted by those who operate the trial. One of the realities of DMC operations involves the real-time analysis of emerging study data that has yet to undergo the full quality-control checks to ensure completeness and accuracy of the data.
However, didactic training and review of case studies, alone, may be insufficient. Effective training of prospective DMC members should also incorporate formal, supervised longitudinal apprenticeships in the setting of actual DMC proceedings, including closed sessions during which the most critical and sensitive issues are addressed. The adoption and endorsement of this type of comprehensive training by sponsors and other key stakeholders will help ensure that a new generation of DMC members is adequately prepared.
To advance this effort, stakeholders with an interest in the role and function of DMCs (e.g. professional, scientific, and medical societies and organizations) should consider developing and maintaining databases of qualified DMC members that include a listing of their experience and relevant expertise.8 In compliance with confidentiality provisions for a given trial, DMC members should also be encouraged to submit interesting and instructive DMC case studies to peer-reviewed journals in order to raise awareness of important issues and challenges that can arise during a clinical trial. Legal and contractual issues concerning service on a DMC (e.g. indemnification) require thoughtful discourse but were not formally addressed in our DMC project.
Discussion
The rationale for using a DMC in clinical trial monitoring is predicated on the need for periodic assessment of the risks and benefits in an ongoing trial guided by a well-defined DMC charter that is aligned with the research protocol and statistical analysis plan. Similarly, our recommended best practices for DMC oversight and communication are intended to ensure the validity and sensitivity of this monitoring process to detect early evidence of avoidable harm, futility, or benefit, and to communicate DMC recommendations in a manner that is actionable when necessary and maintains trial integrity to the greatest extent possible.
An independent, knowledgeable, and well-trained DMC serves the trial sponsor, trial leadership, investigators, and study participants through this periodic assessment of risks and benefits. DMCs have an important and unique role in trial oversight that is substantially distinct from institutional review boards, ethics committees, or trial steering committees, which do not see unblinded interim results. Thus, the role of the DMC cannot be delegated or shared with other entities without the potential for substantially increased risk to trial integrity, and thus also to study participants and sponsors.
The choice of DMC members should be thoughtfully considered, and the role of the chair should never be bestowed on an individual solely by virtue of their position or status in academia or as a key opinion leader. Previous experience acting as a member of a DMC should be a primary consideration, as this experience is invaluable for effectively leading the DMC and providing guidance to newly trained members. Our recommendation for apprenticeship and mentoring necessitates close interaction among DMC team members.
The composition of the DMC is especially important in light of its responsibility to make the best possible recommendations unbiased by the sponsor or commercial interests with relatively sparse information, given that their recommendations often result in irreversible actions being taken. For example, if a trial is stopped and the sponsor and trial leadership is unmasked to treatment assignment, that action cannot be undone. Even if trial enrollment is only suspended for a potential safety concern, it is often difficult or impossible for the prior rate of patient enrollment or investigator enthusiasm to be regained should trial enrollment be resumed.
While our recommendations for DMC use, conduct, communication, and member training form the foundation for improved oversight of clinical trials and enhanced participant safety, it is the effectiveness of the implementation of these recommendations that will determine whether the potential benefits are realized. Several recommendations proposed by us are well aligned with those of the NIH, specifically regarding the importance of DMC access to the unmasked trial data, the need to identify and adequately train new DMC members, and the restriction of attendance at the closed sessions to DMC members only.7 Our recommendations should, ideally, serve as a call to action, encouraging all those involved in clinical trial design and conduct to ensure the DMC structure, charter, membership, and implementation are all consistent with these recommendations. Doing so will ultimately benefit study participants, study sponsors, investigators, and society.
Acknowledgments
The authors wish to acknowledge the contributions of the DMC Project Team and Zachary Hallinan (Clinical Trials Transformation Initiative). Medical writing assistance was provided by Kelly Kilibarda, PhD, in affiliation with Whitsell Innovations, Inc. All authors are considered equal contributors.
Declaration of conflicting interests
The views expressed in this publication are those of the authors and do not necessarily reflect the official policies of the Department of Health and Human Services, nor does any mention of trade names, commercial practices, or organization imply endorsement by the US Government.
Funding
Funding for this manuscript was made possible, in part, by the Food and Drug Administration through grant R18FD005292 and cooperative agreement U19FD003800. Partial funding was also provided by pooled membership fees from the Clinical Trials Transformation Initiative’s member organizations.
References
1. Organization, review, and administration of cooperative studies (Greenberg Report): a report from the Heart Special Project Committee to the National Advisory Heart Council, May 1967. Control Clin Trials 1988; 9: 137–148.
2. SCT Working Group on Data Monitoring, Dixon DO, Freedman RS, et al. Guidelines for data and safety monitoring for clinical trials not requiring traditional data monitoring committees. Clin Trials 2006; 3: 314–319.
3. Calis KA, Archdeacon P, Bain RP, et al. Understanding the functions and operations of data monitoring committees: survey and focus group findings. Clin Trials 2017; 14: 59–66.
4. Clinical Trials Transformation Initiative. CTTI data monitoring committees project expert meeting, https://www.ctti-clinicaltrials.org/briefing-room/meetings/ctti-data-monitoring-committees-project-expert-meeting (2015, accessed 25 July 2016).
5. American Heart Association. Consensus-based decision-making processes, https://www.heart.org/idc/groups/heart-public/@wcm/@mwa/documents/downloadable/ucm_454080.pdf (accessed 5 August 2016).
6. DeMets DL, Ellenberg SS. Data monitoring committees – expect the unexpected. New Engl J Med 2016; 375: 1365–1371.
7. Office of Inspector General. Data and safety monitoring boards in NIH clinical trials meeting guidance, but facing some issues. Washington, DC: Department of Health and Human Services, Office of Inspector General, 2013.
8. Duke Clinical Research Institute (DCRI). DCRI leadership in data monitoring committees, http://dmc.dcri.org/ (accessed 8 September 2016).
Supplementary Material
Files in Data Supplement
