What We Mean When We Talk About EvGen Part II: Building Out a National System for Evidence Generation
By: Rachel E. Sherman, M.D., M.P.H., and Robert M. Califf, M.D.
In an earlier FDA Voice blog post, we discussed a pair of concepts – interoperability and connectivity – that are essential prerequisites for the creation of a successful national system for evidence generation (or “EvGen”). In this post, we take a look at how we would apply these constructs as we go about building such a system.
Building EvGen
Creating knowledge requires the application of proven analytical methods and techniques to biomedical data in order to produce reliable conclusions. Until recently, such analysis was done by experts operating in centers that typically restricted access to data. This “walled garden” approach evolved for several reasons: the imperative to protect the privacy and confidentiality of sensitive medical data; concern about the negative consequences that could arise from inappropriate, biased, or incompetent analysis; and, the tendency to see data as a competitive asset. Regardless of the specific reason, the result has been the same: widespread and systemic barriers to data sharing.
If we are to reverse these tendencies and foster a new approach to creating evidence of the kind envisioned for EvGen, we must bear in mind several critical principles:
There must be a common approach to how data is presented, reported and analyzed and strict methods for ensuring patient privacy and data security.
Rules of engagement must be transparent and developed through a process that builds consensus across the relevant ecosystem and its stakeholders.
To ensure support across a diverse ecosystem that often includes competing priorities and incentives, the system’s output must be intended for the public good and be readily accessible to all stakeholders.What Would EvGen Look Like in Practice?
What would a robust national platform for evidence generation look like? It may be helpful to envision EvGen as an umbrella for all activities that help inform all stakeholders about making treatment decisions.
The task of evaluating drugs, biologics, or devices encompasses different data needs and methods. However, all share a common attribute: the characterization of individuals and populations and their associated clinical outcomes after they have undergone diagnostic or prognostic testing or been exposed to a therapeutic intervention.
Moreover, when medical practice itself is part of the evaluation, characterization of the organization and function of delivery systems is critical. In other words, the kinds of evidence needed to evaluate medical products for safety and effectiveness and the kinds of evidence needed to guide medical practice overlap substantially.
Over the last decade, there has been enormous progress in the area of “secondary use,” in which data collected for one purpose (for instance, as part of routine clinical care) can be reused for another (such as research, safety monitoring, or quality improvement).
The Sentinel Initiative, launched in response to a Congressional mandate to develop an active postmarket risk identification and analysis system, is one example. Modeled after successful programs such as the Centers for Disease Control and Prevention’s Vaccine Safety Datalink, Sentinel allows FDA to conduct safety surveillance by actively querying diverse data sources, primarily administrative and insurance claims databases but also data from electronic health record (EHR) systems, to evaluate possible medical product safety issues quickly and securely.
Another example, the National Patient-Centered Clinical Research Network (PCORnet), is a national system that includes many of the attributes needed for EvGen. PCORnet includes participation from government, industry, academia, and patients and their advocates. Whereas FDA’s Sentinel system is built primarily on claims data repurposed for safety surveillance, PCORnet is designed to leverage EHR data in support of pragmatic clinical research.
The NIH’s Health Care Systems Research Collaboratory has demonstrated through its Distributed Research Network that the concept of secondary data use can be extended into the realm of prospective pragmatic interventional trials. The NIH Collaboratory program, which includes many of the same health care systems involved in Sentinel and PCORnet, has 10 active trials underway.
In addition, the Reagan-Udall Foundation Innovation in Medical Evidence Development and Surveillance (IMEDS) Evaluation Program is exploring governance mechanisms to ensure that private-sector entities, notably regulated industry, can collaborate with Sentinel data partners to sponsor safety queries about marketed medical products. Such measures have the potential to expand the involvement of private-sector partners beyond the arena of methodology, further helping to ensure that Sentinel continues its expansion into a national resource.
Similarly, efforts are underway to establish a National Device Evaluation System (NDES). As currently envisioned, the NDES would be established through strategic alliances and shared governance. The system would build upon and leverage information from electronic real-world data sources, such as data gathered through routine clinical practice in device registries, claims data, and EHRs, with linkages activated among specific data sources as appropriate to address specific questions.
As substantial work already is being done in all of these areas, valuable experience is being gained. The next step is to ensure that these pioneering efforts coalesce into a true national resource. More on that in future postings.
Rachel E. Sherman, M.D., M.P.H., is FDA’s Associate Deputy Commissioner for Medical Products and Tobacco
Robert M. Califf, M.D., is Commissioner of the U.S. Food and Drug Administration
By: Rachel E. Sherman, M.D., M.P.H., and Robert M. Califf, M.D.
In an earlier FDA Voice blog post, we discussed a pair of concepts – interoperability and connectivity – that are essential prerequisites for the creation of a successful national system for evidence generation (or “EvGen”). In this post, we take a look at how we would apply these constructs as we go about building such a system.
Building EvGen
Creating knowledge requires the application of proven analytical methods and techniques to biomedical data in order to produce reliable conclusions. Until recently, such analysis was done by experts operating in centers that typically restricted access to data. This “walled garden” approach evolved for several reasons: the imperative to protect the privacy and confidentiality of sensitive medical data; concern about the negative consequences that could arise from inappropriate, biased, or incompetent analysis; and, the tendency to see data as a competitive asset. Regardless of the specific reason, the result has been the same: widespread and systemic barriers to data sharing.
If we are to reverse these tendencies and foster a new approach to creating evidence of the kind envisioned for EvGen, we must bear in mind several critical principles:
There must be a common approach to how data is presented, reported and analyzed and strict methods for ensuring patient privacy and data security.
Rules of engagement must be transparent and developed through a process that builds consensus across the relevant ecosystem and its stakeholders.
To ensure support across a diverse ecosystem that often includes competing priorities and incentives, the system’s output must be intended for the public good and be readily accessible to all stakeholders.What Would EvGen Look Like in Practice?
What would a robust national platform for evidence generation look like? It may be helpful to envision EvGen as an umbrella for all activities that help inform all stakeholders about making treatment decisions.
The task of evaluating drugs, biologics, or devices encompasses different data needs and methods. However, all share a common attribute: the characterization of individuals and populations and their associated clinical outcomes after they have undergone diagnostic or prognostic testing or been exposed to a therapeutic intervention.
Moreover, when medical practice itself is part of the evaluation, characterization of the organization and function of delivery systems is critical. In other words, the kinds of evidence needed to evaluate medical products for safety and effectiveness and the kinds of evidence needed to guide medical practice overlap substantially.
Over the last decade, there has been enormous progress in the area of “secondary use,” in which data collected for one purpose (for instance, as part of routine clinical care) can be reused for another (such as research, safety monitoring, or quality improvement).
The Sentinel Initiative, launched in response to a Congressional mandate to develop an active postmarket risk identification and analysis system, is one example. Modeled after successful programs such as the Centers for Disease Control and Prevention’s Vaccine Safety Datalink, Sentinel allows FDA to conduct safety surveillance by actively querying diverse data sources, primarily administrative and insurance claims databases but also data from electronic health record (EHR) systems, to evaluate possible medical product safety issues quickly and securely.
Another example, the National Patient-Centered Clinical Research Network (PCORnet), is a national system that includes many of the attributes needed for EvGen. PCORnet includes participation from government, industry, academia, and patients and their advocates. Whereas FDA’s Sentinel system is built primarily on claims data repurposed for safety surveillance, PCORnet is designed to leverage EHR data in support of pragmatic clinical research.
The NIH’s Health Care Systems Research Collaboratory has demonstrated through its Distributed Research Network that the concept of secondary data use can be extended into the realm of prospective pragmatic interventional trials. The NIH Collaboratory program, which includes many of the same health care systems involved in Sentinel and PCORnet, has 10 active trials underway.
In addition, the Reagan-Udall Foundation Innovation in Medical Evidence Development and Surveillance (IMEDS) Evaluation Program is exploring governance mechanisms to ensure that private-sector entities, notably regulated industry, can collaborate with Sentinel data partners to sponsor safety queries about marketed medical products. Such measures have the potential to expand the involvement of private-sector partners beyond the arena of methodology, further helping to ensure that Sentinel continues its expansion into a national resource.
Similarly, efforts are underway to establish a National Device Evaluation System (NDES). As currently envisioned, the NDES would be established through strategic alliances and shared governance. The system would build upon and leverage information from electronic real-world data sources, such as data gathered through routine clinical practice in device registries, claims data, and EHRs, with linkages activated among specific data sources as appropriate to address specific questions.
As substantial work already is being done in all of these areas, valuable experience is being gained. The next step is to ensure that these pioneering efforts coalesce into a true national resource. More on that in future postings.
Rachel E. Sherman, M.D., M.P.H., is FDA’s Associate Deputy Commissioner for Medical Products and Tobacco
Robert M. Califf, M.D., is Commissioner of the U.S. Food and Drug Administration
