It seems like everyone in the life sciences industry is interested in Real-World Evidence (RWE) these days. Companies want to know how they can use it to accelerate approvals. Regulators want to know if it can help them track safety issues. Legislators want to see if it can make drug development more efficient.
And data scientists… they want to know if they can better control for the quality of RWE. Today, we have a fun read for you about a new initiative from the FDA’s Sentinel Initiative, which wants to build a grading scale for “regulatory grade” HER data. AgencyIQ’s Laura DiAngelo has the details for us below.
One more thing – the AgencyIQ team is excited today to announce a new member of our research team: Corey Jaseph, MS, RAC. Corey has decades of experience in medical device regulation and is going to be helping us to enhance our U.S. and E.U. medical device analysis capabilities.
Interested in getting our daily newsletter, FDA TODAY, in your inbox instead of on LinkedIn??You can sign up for free here.
Send questions, comments or requests to hear more about AgencyIQ my way: ([email protected] / @AlecGaffney)
FDA looks to standardize electronic health record data quality
The top line: The FDA’s Sentinel Initiative is looking to build a common protocol to help the FDA assess the quality of real-world data (RWD) from electronic health records (EHRs).
First, let’s talk about the FDA’s Sentinel System and Real-World Data (RWD).
- The Sentinel Initiative is FDA’s real-world surveillance and pharmacovigilance system. Through Sentinel, the FDA uses routine queries to scan safety and utilization of regulated products from electronic health care data. The entities who share data with the system are called “Data Partners,” and include health systems, research institutions and payer systems. According to the FDA’s annual update on the program, in 2020 the Sentinel Distributed Database includes 788 million person-years of data, with 71 million people currently accruing new data in the system.
- Sentinel can support and bolster real-world data (RWD) research projects. The FDA can use data from Sentinel to “strengthen and complement their other data sources,” such as adverse events reporting, study results and trial data, to help inform product safety considerations. In addition, the Sentinel Initiative provides a framework for other investigators to leverage the system, including a Common Data Model that allows Data Partners to “quickly execute distributed programs against local data.” Through Sentinel, researchers and Data Partners can also run research projects – such as exploratory analyses on the use of drugs under REMS ETASU or discussions on using natural language processing (NLP) methods for pharmacovigilance from EHR data – that bolster technological approaches to regulatory issues.
Sentinel’s data partners want to standardize the way that data from electronic health records (EHRs) can be considered of high enough “quality” to be used by the FDA and incorporated into Sentinel.
- The FDA has a new research project for Sentinel: Build a scale for “regulatory grade” EHR data. According to the project announcement , Sentinel’s Data Partners will “create a publicly-available common protocol for Data Quality Metrics for electronic health record (EHR) data.” Researchers will work to define specific metrics of data quality for both inpatient and ambulatory care setting data. Judith Maro and Jeffrey Brown of Harvard Pilgrim Health Care Institute and Harvard Medical School (longtime Data Partners in Sentinel) will lead the workgroup, while representatives from Massachusetts General, Duke, TriNetX and the FDA will also contribute to the project.
- The effort is part of the Sentinel Initiative’s Data Quality Metrics (DQM) System project, which is intended to “provide a harmonized data characterization toolkit to enable researchers to efficiently compare data sources to better contextualize data quality and fitness-for-purpose and to help with the interpretation of findings – to find the right data to answer the question.” The project has defined three basic characteristics related to defining “regulatory grade” RWD: (1) the quality and characteristics of the data itself; (2) the source of the data and institutional characteristics; and (3) fitness for use.
- Real-world data sources, such as data from EHRs, are not built with FDA review in mind. As a result, the FDA has been hesitant to rely on these data. While the data in EHRs does need to be in standardized formats (as required by the Office of the National Coordinator for Health Information Technology (ONC), which sets standards for health information technology products in the U.S.), the inputs are intended to support clinical care – not research. As the new project announcement explains, “since use of EHR data in Sentinel needs to meet FDA data quality standards, creation of a common protocol to assess data quality of new EHR-based data sources supports the FDA priority for creation of a Real-World Evidence (RWE) Data Enterprise.”
- In short, the project will look to create a grading scale that the FDA and Sentinel will use when determining whether EHR data is of sufficient quality to be used for regulatory purposes or built into Sentinel, based on the frameworks established by the DQM projects.
- The concept of an FDA “RWE Data Enterprise” was introduced in 2018 under then-Commissioner Scott Gottlieb, who stated that the agency wanted to “build a modern system that would rely on the electronic health records from about 10 million lives. This system would expand on the data enterprise that we already maintain [i.e., Sentinel and its partners] by incorporating new information from electronic health records… that would allow us to more fully evaluate medical products in the post-market setting.”
Quality and consistency are a big concern for the FDA when it comes to RWD.
- In general, the FDA has been open – albeit cautiously – to using RWD and real-world evidence (RWE) (i.e., the analyses of RWD) to support certain regulation activities – but only when the data is of “regulatory-grade.” However, it has been historically difficult to understand the metrics by which the FDA will find RWD sources to be of sufficient quality.
- Some of these considerations are laid out in the FDA’s 2017 guidance on RWD for medical devices. Notably, the agency has had more experience with leveraging RWE for medical devices than it does for drugs, as RWD sources (especially registries) can lend themselves fairly well to the total product lifecycle (TPLC) approach to medical device regulation. At a high level, the FDA wants any source of RWD that a sponsor is considering using to be both relevant and reliable, even before conducting the analyses to generate RWE in support of a research question.
- Data relevance is how the data actually relates to “the applicable regulatory question or requirement, in part or in whole.” For example, data sourced from Medicare records will be comprehensive for the covered benefits, but also only for the covered population (i.e., those over 65) and would be largely irrelevant for maternal or pediatric research questions. Further, data sourced from a hospital would have limited insight into the use of non-physician-administered drugs filled in a pharmacy.
- Data reliability is considered by the FDA to have two key aspects: accrual and assurance. In order for the RWD to be reliable enough for use, the FDA expects that the source from which the data is collected (accrual) and the processes in place during collection and analysis to minimize errors (assurance) are well established.
- The unique benefits of EHR data for label expansions: Earlier this year, the FDA’s Center for Devices and Radiological Health (CDRH) issued a report outlining almost 100 case studies of the use of RWE in medical device regulation. While the report covers a broad scope of regulatory applications, product types and therapeutic areas, medical record data was typically used to support an expansion of the product’s approved labeling (which would be represented as “off-label” use in a medical chart). Because registries may have limited insight into off-label uses, and payers may have limited willingness to reimburse claims for unapproved uses, EHR data can provide unique insights into these unapproved uses in populations for which products may already be in use.
- Sentinel’s project comes as the FDA has committed to advancing its RWD strategy for drugs. As mentioned above, the FDA’s strategy and familiarity with using RWE for drugs is less well developed than the framework for devices. However, under the FDA’s commitments for the upcoming drug user fee program reauthorization (PDUFA VII), the agency states that it will create an “Advancing RWE” pilot program , in which the agency will review a limited number of RWE-based analysis plans per year, making lessons-learned and best practices from these studies available to the public and using its experiences to update the agency’s RWE guidance documents. FDA has also said that it wants to use Sentinel data to potentially replace certain post-marketing requirements for companies.
What we're watching and reading
- …this announcement from Moderna that they have completed the filing of their Biologics License Application with the FDA for their Covid-19 vaccine (i.e., their application for approval is done and undergoing FDA review).
- …this news that J&J says a study supports a second “booster” dose of its Covid-19 vaccine.
- …this look at ten different ways that the FDA’s Accelerated Approval pathway could be reformed. (Or read the original paper here)
- …this fact-check from the Associated Press about claims on social media that the FDA had approved the drug Thalidomide back in the 1950s. Which is basically the exact opposite of what had happened.
- …this form, DMF 3938, that FDA wants companies to use with all Drug Master File submissions.
- …this upcoming meeting hosted by the FDA on computational science.
- …the FDA’s social media team, which has been on a roll lately.
Analysis available exclusively to AgencyIQ subscribers:
The FDA plans to host a workshop in October on artificial intelligence (AI) and machine learning (ML) enabled medical devices, focusing on how to define “transparency” for these products and the best way to communicate this information to users.
The life sciences industry has expressed overwhelming support for a draft guidance from the FDA that describes when and how sponsors can include Patient Reported Outcome (PRO) data in drug labeling. However, comments also requested additional clarity, modifications and context on most of the points contained in the 11-page draft guidance.
The FDA last week sent a Warning Letter to a Utah-based stem cell company alleging major violations of good manufacturing practices at the facility and that the company is marketing products without FDA approval. Although it is the first such letter to reference the violation following the end to a multi-year enforcement discretion policy, the most notable aspect of the warning may be that FDA is offering specific warnings about different product classes.
The FDA soon plans to reject the submission of electronic Common Technical Document (eCTD) applications if they contain certain validation errors related to the inclusion of specific file elements, it announced today.
The FDA today released a widely anticipated goals letter as part of its efforts to reauthorize the Prescription Drug User Fee Act. While the letter has been slowly negotiated between regulators and industry stakeholders since last year, today is the first time that the public has been able to dive into the details of the proposals. Rather than skim over the details, AgencyIQ has conducted a thorough review of the 71-page document and has a tight summary of key details you need to know.
Today the FDA announced it will hold a public meeting on September 28th to discuss its proposed Commitment Letter for the Prescription Drug User Fee Act reauthorization for fiscal years 2023 through 2027. The agency’s announcement also highlights various “enhancements” included in the commitment language.
Today the FDA released product-specific guidance documents clarifying the development pathways for 37 products, including 23 that had never before received FDA guidance.
CDER plans to launch a new group next month, the Pharmaceutical Supply Chain Governance Board, that will serve as a coordinating body with oversight of “major supply chain issues,” the agency announced today.
