Reconciliations

During the course of doing many audits (and having been audited scores of times), the question of “reconciliation” always arises.  The auditor or inspector asks, sometimes sounding quite “innocent”, whether the company does reconciliations.  The answer is invariably yes.  However, when the auditor starts asking for details, the situation can get quite nasty.

The issues are largely twofold:

First, what exactly is reconciled?  Which databases and data?

Second, how much of the data is reconciled?

What databases must be reconciled?

Most people think of reconciliation to mean making sure that the clinical trial database agrees with the drug safety database.  This certainly is true and indeed these two databases must be reconciled.  Actually this is a bit complicated since the clinical research database usually holds both serious adverse events (SAEs) and also non-serious adverse events (NSAEs).  The safety database usually only holds, from clinical trials, SAEs only (or a few NSAEs which happen to accompany the SAEs in each case).

The problem arises when there are other databases that hold safety data.  Fundamentally any database that is used to collect safety data, where the data is ultimately going to be submitted to a health authority (FDA, EMA etc.), must be secure and validated.  What this means is that the integrity of the database and the data must be assured.  The company must be in full control of the safety data – none can be lost, misplaced, deleted (accidently or on purpose), faked, changed etc. without an audit trail or some other means to ensure its integrity.

This is rarely a problem when a commercial (“shrink-wrapped”) database is used such as Argus, or ARISg or others.  These databases are usually purchased or leased by the company and have accompanying validation documents.  The company should then install the database and do final validation testing (mainly user acceptance testing – UAT but sometimes other testing must be done) and the compliance/IT folks then “certify” that it is ready for use.  Such validated databases have clear audit trails and protect the integrity of the data.

Similarly for the medical/clinical research database, this is often a commercial database that also comes with validation materials and is installed, tested and certified.

Reconciling these two databases is usually straightforward and done either electronically, manually or using some combination of both.  That is, lists can be printed out of cases in each database and compared to ensure no case is missing and that the data is the same in both databases (It’s actually not that simple.  See below.).

Reconciling the databases to be sure that the clinical trial AEs are the same in the two databases is usually done periodically during the clinical trial rather than waiting till the end of the trial.  This corrects errors in near real-time and avoids late expedited reports to health authorities.  It also saves doing a massive amount of reconciliation at the end of the trial when everyone is hurrying to lock the database.

This is all well and good.  However, there are other databases that hold safety data and which most companies may not think of.  For example:

• Databases in the CRO or in the clinical research and safety departments of partners or co-developers/co-marketers and used for governmental submissions (perhaps outside the US).
• The call center database or databases.  Some companies will have multiple call centers in the US and abroad to cover multiple markets or languages.  These databases hold the initial intake information on AEs that are sent (or should be sent) to the drug safety department for handling.
• The electronic data capture system database.  Some products maintain an internal database in addition to the company safety database(s).
• Email databases of individual case safety reports that are emailed or that are received by a fax server and converted into email.  Many companies store these email intake cases (with pdfs and medical records) on an internal drive.  Sometimes the attached documents are printed out and filed, sometimes stored as pdf files in the company clinical trial and/or safety database(s).  Sometimes they and their pdf attachments are left in the email only.
• Regulatory Department databases of expedited reports sent to FDA and other health authorities (HAs).  Sometimes the regulatory department (or whoever sends the cases to FDA, EudraVigilance etc.) keeps their own spreadsheet or database.  This data (along with the case number, version number and submission date) may or may not be also kept in the drug safety database.
• In companies involved in lots of litigation, the law department may maintain a database that includes SAEs.  Hopefully these have all been sent to the drug safety department.
• In some large companies, subsidiaries may maintain local safety databases (often in their own language and often using only a spreadsheet or some other non-validated, home-grown database).
• Clinical research monitors’ databases may contain SAEs that are not sent through the usual channels to drug safety and clinical research.
• The “database” at clinical sites which fax or phone SAEs into the company or CRO.
• Others – many other small databases may be kept to serve various specific functions in a company.

In summary then, there are often multiple databases that contain safety data.  They should all agree and should all hold the same data.  In practice this rarely happens and something slips through the cracks.

Solutions: There are many ways of handling this.  Some companies force everyone to use only corporate issued (or created) validated databases.  Others turn a blind eye to the situation and hope for the best.  The most common solution is to have the IT department and the drug safety group identify all the databases in question and then the decision is made as to which databases must be controlled and reconciled.  This usually includes the clinical research database, the drug safety database, the regulatory database (if there is one) and the call center databases.  The local databases or spreadsheets are often ignored.  This is a practical solution and usually works though it is not at all ideal.  Problems occur if someone discovers data sitting in someone’s computer or local database that was not transmitted for whatever reason to drug safety or clinical research.

Suggestions: The compliance, IT, drug safety and clinical research folks should sit down and make an inventory of all databases that contain safety information.  Then figure out which are critical and must be controlled and validated (clinical research, drug safety, call centers, regulatory, CRO and SAEs faxed or sent in from sites – at the very least) and which are not.  The key issue is to decide which databases might have important safety data that is at risk of not getting into the two key databases: drug safety and clinical research.  Prioritize and determine which databases must be validated and reconciled.  Those which are not in control need to be put in control rapidly.  All of this should be documented to be able to justify to FDA or other inspectors why some databases are validated and controlled and some not.

A clear list of reconciliations should be drawn up and, ideally, put into a working document or some other formal procedural document that can be shown to an FDA or EMA or other inspector or auditor to justify what is reconciled and what is not.

What data must be reconciled?

Once you have determined which databases must be reconciled, the next big issue is to determine what data must be reconciled.  This actually is trickier than deciding which databases need to be reconciled.

Must there be 100% reconciliation al the time?  The purist would say yes.  The realist would say sometimes yes, many times no.

There are lots of technical issues.  For example the clinical research database may hold much more information about dosing than the drug safety database.  Clinical research cares about whether the patient skipped doses, took them at the right time of day etc.  Drug safety does not care as much since they are basically looking at whether the drug was taken at all and start and stop dates.  Missing a dose is usually not a big issue in the pharmacovigilance world.

Other technical issues include multiple identification numbers for the same case (a clinical research patient number, a drug safety number, a subsidiary number (if the case came from abroad), an FDA number, etc.  Some databases capture age and others birthdate and sometimes both (and sometimes the age produces issues if the AE starts before the patient’s birthday and follow up occurs after the patient’s birthday – his or her age has just increased by a year!). Drug names, verbatim codes, misspellings, and all sorts of other issues occur. These issues are well known to anyone who does reconciliations.

Solutions: The usual procedure then is to decide for each set of databases to be reconciled which fields are to be reconciled 100% and which are not.  For example, the identification number, serious/non-serious classification, age, MedDRA codes, co-medications, drugs’ start and stop dates, outcome, etc. should all be 100% reconciled. Other fields (weight, height etc.) are not reconciled. This should be put into a formal procedural document and a process owner must be assigned to ensure that the task is done as specified in the procedure.

Other reconciliations can be less rigorous.  For example, if a site faxes an SAE form to the company, there should be a reconciliation (usually the same day or the next business day) to be sure that the fax was received, that the full x pages were received and were readable.  That is usually enough.

Conclusion: Reconciliations are tricky.  If the correct work is done up front to identify which databases and which data need to be reconciled and a doable process is put in place, this painful process can be made more tolerable.  It will save possible late expedited reports and enormous amounts of work at study end or just before NDA/MA submission (with management breathing down your neck…).