Are Clinical Data Review efforts an overkill or an understated area in Clinical Research?

Any views or opinions represented in this blog are personal and belong solely to the blog owner and do not represent those of people, institutions or organizations that the owner may or may not be associated with in professional or personal capacity, unless explicitly stated. The owner of this blog makes no representations as to the accuracy or completeness of any information on this article.

In Data Management, We often encounter these questions & I am sure every organisation has been pondering about this for a long time now.

1. Do we overdo Data Review specially when it’s proven only ~3-4% data ever changes once it’s entered?

2. Are the efforts & resources we put in worth the output it yields from a data completeness point of view?

In my past experience as a Data Management professional & most recently as a Consultant, I always felt that while both questions had merits of its own if we looked at those independently. The mistake most of us do is to answer one question with the other by finding a correlation. I was trying to elaborate this difference to a client I was consulting recently & thought if I could pick the brains of my industry colleagues by inviting them to share their experience.

To start with I tried to explain 2 basic differences w.r.t Data Validation va Data Review to my client so that we can put things into a better perspective before jumping into conclusions related to CDM efforts.

A. Data Validation:

It’s mostly carried out by help of automated checks to identify data discrepancies or data failing conformance to expected rules. (e.g. Height = 8 ft, Weight = 200 KGs, Age = 121 yrs, Sys BP = 160 mmHg.)

Now most of these discrepancies could be either transcription errors which Sites are able to rectify real time before a Data Reviewer picks it. There are some which are genuine outliers & need further confirmation from Sites if it is left unchanged without a plausible clarification. Therefore a Data Reviewer involvement is much needed in such cases. This is non negotiable unless the data conformance rules itself are questionable.

B. Data Review:

It’s mostly done on data which has passed the so called data validation stage. While most organisations prefer to do it in a continuous manner, some choose to do it in a phased manner on subset of patients or pre-defined data cut-offs. This often requires cross functional alignment with Clinical & Clinical Ops. Data Review involves different levels of clinical/medical judgment & expertise to correlate data outliers with other data values for the patient. The objective is not limited to catching data entry errors. In fact Data Review is carried out by experienced Data Managers or Clinical Team members with good disease area knowledge & analytical skills to identify potential safety signals, protocol deviations, under reported critical data points (e.g. Adverse Events or Concomitant Therapies etc.) leadings to generation of new data which goes through data validation stage again.

Therefore, as a consultant or analyst, I try to present the cumulative efforts that goes into Data Review & not measure it purely with Data Changes once it’s originally entered. The 3-4% figure is highly understated as it doesn’t reflect the effort which goes into discovering new data points. (e.g. A site may confirm the BP value 160 mmHg as recorded in source without changing it at data validation stage, however during data review, one may discover a potential trigger & impact of this data point leading to entry of new medical event(s) or therapies for the patient). If that happens the query has indeed influenced multiple data points across different datasets though the original data point remain unchanged.

In summary, Data Review is often human, & painstaking, sometimes aided by Data Review tools or reports. This may not always result in significant data changes to existing data but discovery of new data & answers to implausible data. The inability to measure data review efforts is indeed an achilles heel for operations. A good data review team cannot be measured by the number of queries they are able to generate or number of checks they do periodically. It often requires discipline & balance of a marathon runner (if I may use this analogy). It means the team is always agile & maintain a steady pace throughout the data review life cycle conserving the energy for the last leap. The reason I say that is at Database Locks, the team is faced with end loading of new data & new perspective as some observations at this stage may trigger a retrospective data review process multiplying the data review efforts.

So next time a consultant/analyst is trying to suggest only 3-4% data changes since it’s originally entered, I suggest to read between the lines. Coming back to the merits of this kind of metric, the organisations indeed have an area or improvement when it comes to prioritising how they wish to execute the data review process using an adaptive approach using data science. I will try to cover that in my upcoming posts.

I would love to hear your comments & feedback as I strongly feel that we need to appreciate the real efforts Data Reviewers put in and listen to ideas on how we can bring efficiencies. Please share your thoughts or write to me at [email protected] if you want to reach out in person.