For many years there was little interest or attention paid to what information coming from academia, industry, individual investigators and other sources on drugs, devices and medical treatments.
Some information was never reported. Some information was partially reported. Some information was reported multiple times.
Some publications and on-line resources included sufficient source data (e.g. clinical trial results, raw data etc.) which would allow the reader of the article to look at the source data and see if the authors’, company’s, government’s, investigator’s (etc.) analyses and conclusions were valid or not.
However, for clinical trials, including safety data, it was rare to be able to look at source data to validate the study. Sometimes this was simply due to lack of space available in the journal; sometimes the authors felt the data was proprietary; sometimes there were more mischievous reasons. Sometimes there was outright fraud requiring the withdrawal of papers that were published.
To see information on fraud and retractions see the article by Brainard and You in the journal Science of October 18, 2018.
“Nearly a decade ago, headlines highlighted a disturbing trend in science: The number of articles retracted by journals had increased 10-fold during the previous 10 years. Fraud accounted for some 60% of those retractions; one offender, anesthesiologist Joachim Boldt, had racked up almost 90 retractions after investigators concluded he had fabricated data and committed other ethical violations. Boldt may have even harmed patients by encouraging the adoption of an unproven surgical treatment. Science, it seemed, faced a mushrooming crisis.
The alarming news came with some caveats. Although statistics were sketchy, retractions appeared to be relatively rare, involving only about two of every 10,000 papers. Sometimes the reason for the withdrawal was honest error, not deliberate fraud. And whether suspect papers were becoming more common—or journals were just getting better at recognizing and reporting them—wasn’t clear.”
See also the website for Retraction Watch at www.retractionwatch.org. This website tracks withdrawals of publications but also plagiarism and other bad actions such as failure or delays in notifying patients of data breaches, ignoring regulatory requirements, false email contact addresses in papers (18% according to one article). They note one publisher had over 7000 retractions in the electronics industry. Retraction Watch publishes monthly withdrawal updates. This site is well worth looking at. They cover pharma as well as other industries and groups.
One of the ways to evaluate reporting is via the federal website clinicaltrials.gov which was created in 2007 by the FDAAA and which was finally finalized in 2017. This law/regulation requires that sponsors and investigators doing work on some FDA-regulated products must report status and results (efficacy and safety) on this website.
Unfortunately, FDA did not track compliance. However, various others do track compliance. See FDA Trials Trials Tracker from the University of Oxford at https://fdaaa.trialstracker.net/. They have an interactive module that gives details of reporting with several good filters to pull up subcategories. Read also their very informative FAQ on the website.
This tracker, as of this writing in May 2019, has noted that of approximately 20,000 clinical trials on the site, some 3200 were reported. Many others are ongoing, delayed etc. Of these 1105 were overdue (some by over 480 days), 1440 were reported and 702 were reported late (some again by over 400 days). Overall there was 66% reporting. FDA could have imposed fines totaling over $3 billion but did not.
A recent editorial in the Annals of Internal Medicine looked at this issue: “Not Reporting Results of a Clinical Trial is Academic Misconduct”. The writers discuss this issue and reference to another new publication by Tatsioni et al entitled “Lost evidence from registered large long-unpublished randomized controlled trials: a survey”. Ann Intern Med 2019. [Epub ahead of print].
The Tatsioni article “underscores growing concerns that reporting of clinical trial results does not meet legal and ethical standards. The authors identified the 500 largest clinical trials without reported results or publications registered on ClinicalTrials.gov. They found that these trials, which were unreported for 4 or more years after completion and had enrollments ranging from 511 to 11 000 participants, were unlikely ever to be published or to report results on ClinicalTrials.gov.”
The authors of the Annals editorial further note that before the FDAAA, fewer than ½ of registered and completed trials were published, even those of NIH. Academic center reporting was between 16% and 55% from 2007 to 2010. Large trial (over 500 participants) reporting before 2009 noted that about 25% did not post results.
The editorial notes that, obviously, this is a bad thing and that new approaches need to be put in place to rectify the situation:
- Non-reporting should be considered academic misconduct with penalties such as suspension and non-promotion of investigators who fail to report
- Funding agencies such as NIH should not support researchers who fail to report
- Investigators who fail to report should be prohibited from applying for new grants and current grants should be suspended
- Hold the institution itself responsible
- Publicize the failures to report
The editorial concludes that:
“The conduct of research in humans comes with inviolable responsibilities, including the commitment to share what has been learned. No reason exists for the topline results of a clinical trial not to be made public… (this) may lead to misconceptions about the efficacy and safety of interventions. The time has arrived to address this threat to trust and science.”
Comments
It is hard for anyone to disagree with this. Reporting safety and efficacy is required by law for many drugs. Not only is this a legal responsibility but also an ethical one. The government has not yet really conveyed to the stakeholders that this really must be done.
If one looks at the history of adverse event reporting in post-marketing and clinical trial situations one notes the evolution over decades starting with no required reporting of safety issues, to some required reporting (by sponsors, companies and investigators) to timely (e.g. 15 calendar days) reporting in specific formats with specific information.
This required strong governmental requirements which were ultimately backed up by audits, quality checks, unannounced inspections etc. It has largely worked, by most reckonings, in the area of expedited IND and NDA reporting in clinical trials and in post-marketing situations, particularly for toxic drugs or in patients with significant underlying disease.
The industry, the FDA and other health authorities, have set up complex mechanisms to handle safety reporting with costs of pharmacovigilance running now into billions of dollars annually.
But this has not yet been applied to clinical trial reporting. We are many years behind the current situation in post-marketing safety. However, the major players in this area such as the FDA, the EMA and other health agencies are becoming more and more aware of the problems and are looking at ways to improve the situation. Some companies too are reporting well. Others not. The usual constraints of money, personnel, non-standardized IT and reporting requirements must be dealt with as well as the fact that many trials are international or even global (covering all continents). How this will be worked out remains to be seen. The history of global cooperation in these areas is not always positive.
PV and Drug Safety
Those of us in these domains understand the requirements and obligations (legal and ethical) of safety reporting. Most (many?) of the sponsors and investigators and personnel in this field are fully motivated to comply. But as we all know there are always pressures to delay reporting or shade certain clinical issues in a particular way which may not be fully justified (Is the drug guilty until proven innocent or innocent till proven guilty?).
Those of us who lived through the evolution of the current PV world (e.g. before the EMA was created, before MedWatch etc.) are likely to say that history will repeat itself here. We will see governments paying more and more attention, particularly after some disaster in one or more clinical trials, to tightening the system. This will take years and much money. With a multi-polar world though, the global evolution may be harder than it was with ICH (The old International Conference on Harmonization) when there were really only 3 countries/regions (US, EU, Japan) setting the rules and course.
We live in interesting times.
