Terminology: Signals, Potential Signals, Risks, Identified Risks and Potential Risks

Periodically I write a lamentation posting.  Here is another one.

There is a lot of confusion around now about how to classify signals and risks.  This is leading to differing ways of classifying, handling and reporting safety issues.

Unfortunately, there does not seem to be a standardized classification or definition of all of these terms.  The best we have so far is the report published in 2010 by the CIOMS Working Group VIII entitled “Practical Aspects of Signal Detection in Pharmacovigilance”.  https://www.cioms.ch/index.php/2012-06-10-08-47-53/working-groups/working-group-viii The do an excellent systematic review of signal detection and the handling of risk and their consensus definitions seem to be the best use of terminology for now.  In addition there are some definitions from the EMA noted below.

Here are some of the definitions.

Signals

Signal: “Information that arises from one or multiple sources (including observations and experiments), which suggests a new potentially causal association, or a new aspect of a known association, between an intervention and an event or set of related events, either adverse or beneficial, that is judged to be of sufficient likelihood to justify verificatory action.” This is the CIOMS definition that EMA has adopted.

There are various subtypes of signals or subcategories of signals:

• Regulatory Signal: a signal which includes one or more of the following: unexpected AEs/SAEs, changes in severity, characteristics or frequency of expected ARs, changes in outcomes of known ARs.

• Indeterminate Signal: A signal that has been or is being worked up for which there is no conclusion in regard to whether it is verified (validated) or refuted (closed).

• Verified or Validated Signal: A signal which has been worked up and found indeed to be real, a true risk.

• Refuted Signal: A signal after a signal work-up which has been found not to be a risk or potential risk.  That is, the signal was “false”.

• New Signal: A recently discovered signal.

• Ongoing Signal: A signal that has been known for some time and which is being worked-up, tracked, undergoing evaluation etc.

• Closed Signal: A signal for which an evaluation was completed. This is from the EU Good PV Guidelines Module 7: PSURs page 21.

• Potential Signal: I could not find a formal definition of this.  If any reader does, it would be appreciated if you could share this with us.  FDA talks about it.  See below.

To make matters worse, in Module 7 page 21, the EMA states in a footnote: “In the EU-regulatory network and for the purpose of the PSUR, the term “signal” in this section corresponds with the term “validated signal” described in Module 9-Signal Management.

Now interestingly, FDA appears to have created a new category of signal (a “potential signal” or “potential signal of a serious risk”), which they do not seem to define formally anywhere but which is explained on its “Potential Signal” website (https://www.fda.gov/drugs/guidanceComplianceRegulatoryInformation/Surveillance/AdverseDrugEffects/ucm082196.htm). They also seem to use the terms “potential safety issue” and “potential risk” synonymously.

Here is some of the text from the FDA’s website:

“Title IX, Section 921 of the Food and Drug Administration Amendments Act 2007 (FDAAA) (121 Stat. 962) amends the Federal Food, Drug and Cosmetic Act (FDCA) to add a new subsection (k)(5) to section 505 (21 U.S.C. 355).

This section in FDAAA, among other things, directs FDA to “conduct regular, bi-weekly screening of the Adverse Event Reporting System [AERS] database and post a quarterly report on the Adverse Event Reporting System Web site of any new safety information or potential signal of a serious risk identified by Adverse Event Reporting System within the last quarter.” When a potential signal of a serious risk is identified from AERS data, it will be posted in the required report in the quarter in which it is first identified. A potential signal of a serious risk may in some cases constitute new safety information as defined in FDAAA (newly created section 505-1(b)(3) of the FDCA) which includes, among other things, information derived from adverse event reports about a serious risk associated with use of a drug that FDA has become aware of since the drug was approved or, for drugs that have REMS, since the REMS was required or last assessed. FDA will post each potential signal of a serious risk in the quarter in which it is first identified. If additional new safety information is developed concerning a potential signal that has already been posted, it will be addressed by FDA in new safety communications, but will not appear again as a new quarterly posting.”

The FDA has created, on this website a listing of these “potential signals of serious risks” and states:

“The appearance of a drug on this list does not mean that FDA has concluded that the drug has the listed risk. It means that FDA has identified a potential safety issue, but it does not mean that FDA has identified a causal relationship between the drug and the listed risk. If after further evaluation the FDA determines that the drug is associated with the risk, it may take a variety of actions, including requiring changes to the labeling of the drug, requiring development of a Risk Evaluation and Mitigation Strategy (REMS), or gathering additional data to better characterize the risk.”

The EMA has published, as part of the Good PV Modules a Module on Definitions.  However it is incomplete in terms of defining signals. https://www.ema.europa.eu/docs/en_GB/document_library/Template_or_form/2013/05/WC500143294.pdf

So what we have here is the use of terminology that is not standardized and has created significant confusion in the PV world, particularly when US companies try to differentiate between “signals” and “potential signals”.  What we see is a hodge-podge of definitions.  For example, potential signals are sometimes defined as “weak signals” where there is not much information or little convincing information.  Others define potential signals as an “early” signal which is being looked at and may advance to a “true” signal (or just a plain signal) if the data becomes more convincing.  Some companies, particularly in clinical trials, feel obliged to tell FDA and other regulatory agencies of signals and/or potential signals; other companies only tell FDA and the HAs about signals but not potential signals.  This is all rather confusing and does not seem to advance the public health.

Risks and Safety Concerns

The next area of confusion seems to be in the definition of “risk” and “potential risk” and “safety concerns”. In the Definitions Module, the EMA defines some terms:

• Potential Risk: “An untoward occurrence for which there is some basis for suspicion of an association with the medicinal product of interest but where this association has not been confirmed…Examples include a signal arising from a spontaneous adverse reaction reporting system.”
• Identified Risk: “An untoward occurrence for which there is adequate evidence of an association with the medicinal product of interest.”
• Important Identified Risk and Important Potential Risk: “An identified risk or potential risk that could have an impact on the risk-benefit balance of the product or have implications for public health…What constitutes an important risk will depend upon several factors, including the impact on the individual, the seriousness of the risk and the impact on public health. Normally, any risk that is likely to be included in the contraindications or warnings and precautions section of the product information should be considered important.”
• Important Missing Information: “Critical gaps in knowledge for specific safety issues or populations that use the marketed product.”
• Safety Concern: “An important identified risk, important potential risk or important missing information.”
• Safety Issue: Not defined explicitly in the definition or PSUR modules and not commonly used.

Comments

If you think this terminology is inconsistent, overlapping, repetitive and ugly, you are correct.  The definitions, were originally created in ICH and then CIOMs and then adopted by various health agencies and have been changed or modified along the way.  The US definitions are almost non-existent.  FDA talks of potential signals whereas most others talk of signals which are then worked up and either validated or refuted.  So FDA’s definition of potential signal seems to fit that of just plan vanilla signals and FDA then notes that after work-up signals may be refuted and removed from the list or found to be real (validated) and will result in some action such as a label change.

So what is to be done?  My suggestions are to move to simplicity and to use the EMA terminology in most cases since companies must submit PSURs, PBRERs, RMPs and other documents using their terminology.

• I’d suggest getting rid of the term “potential signal” and using just the term “signal”.  For FDA purposes, a potential signal = a signal.
• For risks, I’d use the three EMA terms: potential risk, identified risk and missing information.  All of these equal “safety concerns”.
• Safety issues seems to be a synonym for safety concerns.

Translation into PV practice

Companies and health agencies should (and do) have teams of experts looking at AEs, SAEs, medical errors, medical literature and more to identify “bad stuff” about their marketed and clinical trial products.  Each “bad thing” should be called a signal and put on a list, tracked and worked up.  I would not differentiate between potential signal or signal but would classify each signal along the lines of weak to strong such as a 1 (weakest) to 4 scale (strongest) or “weak, moderate, strong” or some similar system to give an idea of how much data supports the signal.

All signals need to be evaluated and worked up to some degree.  Some signals will get major workups and others will simply be observed over time.  Priorities can be set depending upon how serious/severe the problem is, how many people are at risk, ability to get information etc. The classification of weak to strong should periodically be re-evaluated and changed if necessary.

Signals must then be translated into “risks” or “safety concerns” for EMA/EU reporting. This is important because such classifications are required in the PSURs, PBRERs and RMPs. This classification, using perhaps different wording, is equally applicable in the US where the formal classification into identified or potential risks is not done explicitly but companies report “bad things” to FDA in varying formats and FDA itself identifies “potential signals”.

Companies should then write their SOPs to indicate what will be done with each classification or signal or concern or risk and then implement the procedure.

One might argue that the EMA procedure is incredibly detailed.  Anyone who has written an EMA PBRER or RMP can attest to this.  The FDA system of signaling and risk management is far less detailed and PBRERs and RMPs are not required.  REMS are much simpler and less detailed as a rule.  Where is the happy medium?  Probably somewhere in the middle with simple definitions and global standards.

So where is this going?  One suspects that there will be more clarity and perhaps a more consistent international classification and methodology on signaling and risk work up as techniques develop.  The CIOMS VIII report is an excellent start and one can easily imagine another international group looking at this in the next several years.  That is how medicine and science work.