In late December, FDA released a new and important draft guidance on proposed changes to safety surveillance and reporting for clinical trials.
This is a very interesting and controversial document and, if put into effect, promises to produce some major changes and issues in clinical research. It is a long and complex document. I will highlight just some of the issues and will then make a few comments and suggestions.
The aim of this draft guidance is to explain how sponsors should develop a systematic approach to IND safety reporting, in particular for situations where there are multiple studies of the same experimental product. The rationale is for sponsors and FDA is to detect and evaluate, as early as possible, SUSARs and clinically important increased rates of known SAEs.
The document has five major themes:
- Use of a Safety Assessment Committee
- Performance of aggregate analyses to compare AE rates across treatment groups
- Planned unblinding of safety data
- Reporting thresholds for IND safety reporting
- Development of a Safety Surveillance Plan
The document then summarizes the IND safety reporting requirements (21CFR312.32, 310.305, 314.80, 600.80 and 606.170). This draft guidance is a fine, concise review of the basic safety reporting requirements and is worth the read and review.
FDA recommends sponsors develop a Safety Surveillance Plan and a Safety Assessment Committee for experimental drugs, even marketed ones undergoing new indications.
Safety Assessment Committee (SAC)
This committee should oversee the evolving safety profile by reviewing the cumulative SAEs from all trials with the product plus other safety information (e.g. animal studies, epidemiology) as well as doing unblinded comparisons of event rates in study drug and control groups.
The goal, interestingly, is to make a recommendation to the sponsor as to whether an event or group of events meets IND reporting requirements. The SAC may make a recommendation to the sponsor regarding “whether any numerical imbalance in the unblinded rates meet the criteria for IND reporting.”
The other duty could be to assist others dealing with the trials’ safety (DMCs, steering committees, etc.) about changing the trial, revising consent, etc.). The roles and responsibilities of the SAC “should be clearly defined and distinguished from the roles of other groups.” The SAC should also review and recommend whether other data (e.g. epidemiologic studies, pooled analyses, animal trials, etc.) should be reported to FDA.
Comment: This is more than just recommendations for expedited reporting to FDA. It is, de facto, signaling.
The SAC should meet periodically at regular intervals and with the possibility of ad hoc meetings if needed.
Next there is a discussion regarding the difference between the SAC and a Data Monitoring Committee (DMC). The reports generated for the DMC may also be used for the SAC but the sponsor should “limit the unblinded data to those data that are necessary to evaluate the event (e.g. reports are modified to exclude efficacy)…”
The DMC monitors the trial for modification or stopping of the study. The SAC would not do this but rather, in theory, determine IND safety reporting to FDA and investigators. The DMC threshold is “generally higher than the threshold for reporting potential risks” to the sponsor.
The SAC can have a variety of compositions and structures. There should be at least one physician familiar with the therapeutic area as well as “clinicians” with general or specific safety experience. Others disciplines may be members (permanent or ad hoc) including epidemiologists, pharmacologists, toxicologists, chemists, biostatisticians etc. For studies of a marketed drug, in individual involved in evaluating post-market safety should be included.
The SAC should not have individuals directly responsible for the conduct or analysis of the trials.
The SAC may be from the company’s organization or may be independent (third party) or a mixture of each.
Safety Assessment Practices
Various reports and analyses are recommended (some of which companies may already do and others which will be new).
- Anticipated SAEs: These are AEs that are known consequences of the underlying disease, events common in the study population (e.g. cardiac events or hip fractures in the elderly), events “known to occur with drugs administered as part of a background regimen” (e.g. neutropenia with myelosuppressive chemotherapy), AEs expected in the population due to age, geographic location etc.Anticipated SAEs from the underlying disease or common in the study population are considered “unexpected AEs”. However, these do not warrant IND 15 day safety reports because “it is not possible, based on a single case, to conclude that there is a reasonable possibility” that the SAE was caused by the study drug. These are not SUSARs. They would, however, be reportable if they were occurring more frequently in the drug treatment group compared to the control group.“At the time of protocol development, the sponsor should identify, in the safety surveillance plan, the anticipated serious adverse events that it does not plan to report individually in an IND safety report under § 312.32(c)(1), together with a plan for monitoring the events.” The SAC should monitor these events and make a recommendation to the sponsor regarding submitting an IND safety report if an aggregate analysis suggests they are occurring more frequently in the drug group.
- Aggregate Analyses of Safety Data: FDA notes that 312.32(c)(1)(i)(C) requires an IND safety report when an aggregate analysis of specific events indicates these occur more frequently in the drug treatment group. The aggregate analysis should be performed across multiple studies. Review of these analyses is complex and is “not a simple application of a planned statistical analysis.” FDA notes that in the past sponsors often compared aggregate analyses to historical comparators, external controls, registries, class labeling etc. Some sponsors used statistical tools. But “the majority of sponsors reported not reviewing unblinded data for imbalances in event rates…for blinded studies.”
The FDA clearly states here that the SAC should review unblinded data to assess rates for SAEs prespecified in the premarket safety surveillance plan as anticipated SAEs or previously recognized SAEs noted in the protocol or IB. “Given the uncertainty of predicted rates…and the substantial challenges of specifying a predicted rate…the preferred approach is to regularly perform unblinded comparisons.” Nonetheless, sponsors should try to specify in the surveillance plan predicted rates of anticipated and previously recognized SAEs and provide guidelines for determining that an observed rate exceeds the predicted rate and informs a determination that the event is causally related.”
In addition, some SAEs will occur that were not specified in the safety surveillance plan and are not interpretable as single events. Yet the sponsor must evaluate them to determine whether they warrant IND reporting. Thus the SAC should evaluate unblinded aggregate analyses for these SAEs.
Comment: The SAC thus should evaluate, in effect, anticipated and unanticipated SAEs using aggregate analyses. This will be to determine if there is an issue/signal that should be reported to FDA. This is, in fact, signaling.
Unblinding
FDA states that it “does not believe that unblinding single or small numbers of SUSARs will compromise the integrity of the study.” FDA also notes that for examining rates of treated and control groups, unblinding must be done to make a determination.
“Sponsors should have appropriate procedural controls and processes for unblinding safety data described in the safety surveillance plan…restricting the number of individuals who have access to unblinded data (i.e., the SAC) as well as a plan to unblind only those data that are necessary to evaluate the event (i.e., treatment assignment of the subjects who experienced the serious adverse event under review, clinical data that may correlate with the event [e.g., serum creatinine for the SAE of acute kidney injury]). Study endpoints, efficacy data, and other data collected for the study that do not pertain to the adverse event should not be unblinded.” Those seeing this unblinded data should remain blinded to the “overall data.” If the sponsor has significant concerns, an alternate method may be proposed to FDA.
Comment: Easier said than done.
Thresholds for IND Reporting
Clearly determining such thresholds for reporting is not easy and will vary from situation to situation. Factors to consider include: size of the difference in frequency, consistent increase in multiple trials, preclinical findings, dose response, plausible mechanism of action, known class effect, occurrence of other related AEs (e.g. strokes and TIAs).
The SAC should re-evaluate data over time to see if the or a threshold has been met based on the new data. Follow-up information should also be reviewed a appropriate if it changes the diagnosis of an AE, death occurs due to the AE etc.
Developing Safety Surveillance Plans
Companies should develop safety assessment plans for their study drugs. These plans should include:
- The Plan should describe the SAC (members, meetings, data needed, operational aspects, unblinding etc.).
- Roles and responsibilities for the SAC, its members and any other parties or committees (e.g. Adjudication Committees, Steering Committee, DMC etc.).
- List of SAEs the sponsor does NOT plan to report as 15-day reports.
- List of previously recognized serious adverse reactions (note: reactions not events) that will be monitored.
- A defined process for routine and timely review of SAEs (note events not reactions) and other safety information.
- Guiding principles for periodic aggregate safety reviews, particularly of unblinded SAE rates.
- Predicted rates of anticipated SAEs and previously recognized serious adverse reactions (expected events).
The Plan should be maintained by the sponsor and available for FDA inspection. Before starting phase 2 or 3 studies, the sponsor should submit a “portion of the plan” to the IND, particularly, a list of anticipated SAEs and previously recognized serious adverse reactions and “guiding principles” for periodic aggregate safety reviews.
Comment: Not sure what a “portion” means here.
The protocol should include a summary of the Plan as well as any study-specific differences from the safety surveillance plan, including any study-specific plans for monitoring specific anticipated serious adverse events in the aggregate.
My Comments
This is a complex document establishing the “requirement/best practice” of creating a formal internal or external or mixed SAC to monitor on-going safety. Their focus is on which SAEs/SARs/SUSARs meet FDA reporting requirements for 15-day (or conceivably even 7-day) reporting. The judgment will be based on aggregate analyses primarily though individual cases may matter too. In fact, this is a signaling committee.
There are many issues here:
The boundaries and responsibilities between the SAC and DMC are hazy.
In some cases DMCs do not receive efficacy data but only safety data. Thus, in effect, the two committees largely overlap. Those who have served on DMCs (including myself) often note that it is hard to make a safety judgment in the absence of efficacy data. That is, a benefit/risk analysis is hard to make if only the risk part of the data is known. This tends to make DMCs more conservative since they cannot balance SAEs against benefits in bad diseases or critically ill patients.It is not clear how the two committees will interact or what will happen if there is a disagreement between them.
Thought experiment: Suppose the SAC recommends an SAE/safety issue be reported to FDA. Suppose the DMC, which sees the efficacy data also, feels that the rates of the event do not meet the (hard to determine) threshold. The implication is that looking at the efficacy data suggests this SAE is not a significant clinical problem. How will the DMC communicate that they are “less worried” than the SAC? How will situations like this be adjudicated? The DMC will have to be very careful in what it says to the SAC to avoid accidently breaking the blind.
The implication here is that aggregate analyses driving an IND safety report are more “valid” and “meaningful” than the reporting of an individual case (ICSR) 15-day safety report. It is now a “real” signal. Since efficacy is not looked at by the SAC, the DMC may now be “forced” to look at the efficacy data for the SAE and come to a conclusion regarding its validity and whether the study should be changed, stopped etc.The timing for these reviews will require urgent coordination, particularly, for fatal or clinically severe SAEs or safety issues.
Potential legal liability should be evaluated for members of the SAC, the DMC and the sponsor. Who knew what when? Were risk minimization procedures put in place rapidly?
The operational mechanism of keeping the SAC separate from the clinical team doing the trials is not going to be feasible in many small and mid-size companies.
“Partial unblinding” is very difficult. It is likely that small and mid-size companies will out-source this function though they may not be happy doing this. No company likes to lose control of its products.
Will the IND safety reports sent to the investigators and IRBs be blinded or unblinded?
Will the investigators or sponsor team be able to “guess” which group is which? This may be possible in some studies depending upon how randomization is done. There are a lot of other operational details that will need to be worked out.
Why are two committees needed? Why not expand the DMC to review both reportability to FDA in addition to their current responsibilities. From an operational point of view this would seem to be much easier than creating this complex of two committees. I think we will see a lively debate about this draft guidance.
What should companies do now?
For those companies doing clinical trials, evaluate your mechanism of trend analysis, signal generation, risk management (or whatever you call it). Be sure there is a system in place to track all SAEs and probably non-serious AEs too in a blinded fashion. Consider creating blinded aggregate reports if multiple studies are underway.
If you do not have a DMC/DSMB in place, even for phase I studies where it is practical, consider setting one up. Strongly consider giving the committee unblinded data if they do not receive it already. Strongly consider giving the committee not just safety data but efficacy data too so that they can do a benefit/risk balance. In my experience, if one only sees safety data, the committee will be more conservative in its judgments. If they can balance safety issues with efficacy data (showing some benefit) they will be able to make their judgment based on benefit/risk not just risk. Start thinking about how you would implement an internal or external SAC if this comes to pass as this draft guidance describes. Follow this draft guidance and see what the final outcome is.
