Two Decades Later: Confusion Still Abounds Over the P1,P2 Approach for Risk Estimation
It was in 2007 that the 2nd edition of ISO 14971, the International Standard for application of risk management, first introduced the concept of P1 and P2 as a way to estimate the probability of occurrence of harm (P) associated with the use of a medical device.
Nearly 20 years later, confusion still abounds in the medical device industry.
It is a very simple and mathematically sound concept. Yet, risk practitioners have continued to be challenged in correctly applying it when estimating risk.
This article offers a quick review of the fundamental concepts in probability to help build a better understanding for practical applications.
In a recent LinkedIn poll , less than half of the respondents answered this simple question correctly:
The correct answer is 1 in 5000.
In this scenario, we are given the information that an individual risk of harm is considered to be acceptable at a rate of 1 in 10,000. The S=5 (Severity level 5) is irrelevant information. We are also given the information that P2 has been estimated to be 0.5.
Applying the formula P (of harm) = P1 * P2; we can easily estimate the maximum tolerable P1 by dividing 1/10000 by 0.5.
However, 39% felt there was insufficient information to answer this question and 9% got confused by the "As far as possible" criteria for risk acceptability.
Let us review the fundamentals
According to Figure C.1 in Annex C of the current 3rd edition of ISO 14971:
P1 = Probability of a hazardous situation occurring
P2 = Probability of a hazardous situation leading to harm
Further; P, the probability of occurrence of harm is related to P1, and P2 by the following equation:
P = P1*P2
There are 2 additional fundamental concepts we need to understand:
Therefore, occurrence of harm is contingent on the occurrence of a hazardous situation.
Let us apply basic probability to understand the relationship between P and P1, P2
A basic concept in probability is that the probability of occurrence of an event (say A) is given by the following equation
P(A) = X/T; where
X = Number of ways event A occurs; and
T = Total number of elementary outcomes
Unless we have a basic knowledge of the process, probability is estimated using a sample of observations. This is also known as empirical probability, which is different from the a priori probability, where we have a fundamental understanding of the process, for example, a coin toss or drawing a card from a standard deck.
Let us now consider our specific situation related to the probability of occurrence of harm (P) and how it is related to P1, and P2. One way to visualize this situation, and associated probabilities is by using a decision-tree like diagram as shown in Figure 2 below:
In Figure 2 above,
Before we go into the mathematical relationship between P, P1 and P2, let us consider the following hypothetical example.
Let us say we have good quality data for a medical device from the field where it was used in 1000 surgical procedures:
Using the data in Figure 3, we can easily estimate some of the simple probabilities illustrated in Figure 2.
What about the other probabilities shown in Figure 2, for example P(B|A), or P2?
This is also called a conditional probability, because the probability of event B (i.e. a harm), in this case, is contingent on the probability of event A (i.e. a hazardous situation). We know this to be a key principle of hazard/harm relationship as noted above.
领英推荐
Notice the following data points in Figure 3:
Therefore, we can use the basic probability equation to estimate this conditional probability:
P(B|A) = 10/100 = 0.1
Note that it is only by a coincidence that P(A), i.e. P1, and P(B|A), i.e. P2 are both equal to 0.1. This is an artifact of the data and not because of any fundamental relationship between the two.
Notice an interesting scenario above:
Therefore, we can also use the following equation for P(B|A):
P(B|A) = Total count of harm events/Total count of hazardous situations
=> P(B|A) =P(A and B)/P(A) = (10/1000)/(100/1000) = 0.1
We can now establish a mathematical relationship between P, P1 and P2
=> P(B|A) = P(B)/P(A); or P2=P/P1
=> P(B) = P(B|A) x P(A); or P = P2 x P1; or P = P1 x P2.
Wait a minute, what about the Bayes' Theorem?
You may have come across the following equation in the context of P(B|A):
=> P(B|A) = P(A|B) * P(B) /P(A)
This is also known as the Bayes' theorem, which is essentially a rule that can be used to revise our probability estimates of an "effect" given some prior information about relevant factors, or "causes". As an example, if we know that a patient's age (or a comorbidity) is a factor affecting the probability of occurrence of harm, we can revise our estimated probability with this a priori knowledge.
We don't need to use the Bayes' theorem to develop the P = P1xP2 relationship as a general rule.
However, the equation above still works. Let us take a closer look:
=> P(B|A) = 1 * P(B)/P(A)
=> P(B) = P(B|A) x P(A); or P = P2 x P1; or P = P1 x P2.
Therefore, there is no need to be confused about the application of the Bayes' rule as it relates to the fundamental relationship between P, P1 and P2.
In a future article, we will review a few applications of the Bayes' rule to illustrate how the probability of occurrence of harm can be revised based on a priori information such as patient's age, comorbidities etc.
In conclusion
Nearly 2 decades after the first introduction of the P1, P2 approach in ISO 14971 for estimating the probability of occurrence of harm, confusion abounds in its practice in the medical device industry. One potential reason is that application of basic concepts in probability can be challenging without a deeper understanding of fundamental principles. This article provides an overview of these key concepts to help facilitate correct application in the industry.
References
Like this article? Click here to unlock exclusive subscriber-only content.
Head of Global Medical Affairs at Smith + Nephew and Senior lecturer at University Hospital Hamburg Eppendorf
1 个月Thank you, Naveen Agarwal, Ph.D. This article hits right into the center of ongoing discussions between Medical and Quality teams. I am glad you were able to phrase my concerns in such an elaborate way. I fully agree with your assessment of P1 and P2 and how these are combined vs. use of arbitrary tables merging P1 and P2 without foundation. I look forward to getting into more detail in HOW to apply P1 and P2 with reason and how you propose to actually merge P1 and P2 - in particular, when there is a lot of uncertainty. In short: how do you create a meaningful matrix when using qulitative or semi-quantitative measures for P1 and/or P2 as proposed by ISO/TR 14971
Medical Product Consultant at Aspen Medical Risk Consulting
1 个月If P1 andf P2 are partisioned properly, IMO, P1 and P2 are the best things that go into ISO 14971 because it can provide a cultural divide between egos and approaches that often don't mix well: Physicians and Engineers. I always define P2 to be the portion of the sequence of events leading to harm that starts when a user touches the product in preparation for using it to perform therapy. This makes 'P2' a medical estimate of the frequency of harm, and leaves 'P1' to be an estimate of the frequency of everything that occurs up to when the hazardous situation occurs (i.e., up to when a user touches the product for therapy). Each 'culture' has their own estimate to contribute to the probability of harm, and they each get to play in their own sandbox.
Medical Product Consultant at Aspen Medical Risk Consulting
1 个月IMHO, the primary problem with undertanding P1 and P2 is the fact that P1 is the probability of the Hazardous Situation and there is a tremendous amount of variability built into the definition for when a Hazardous Situation occurs. While scrupulously following the definition of a Hazardsous Situation in ISO 14971, we can have a long sequence of events leading to harm and any one of those events can be designated to be the Hazardous Situation - anything from the root cause to the actual occurrence of harm. That's a lot of variability for people to process and choose intelligently from.
BioPharma Operations SME at Exyte
1 个月On further refection, both P1 and P2 could be risk mechanism probabilities described by the system risk structure (SRS) shown below,. Pa = probability device will be used - likely certain = 1 Pb = probability of device failing (Note Pb = P1 if Pa = 1) P = harm to patient P1 = probability of device failing P2 = probability of device failing resulting in harm to patient Both P1 and P2 are subject to failure modes that might impact their probabilities.
BioPharma Operations SME at Exyte
1 个月I deleted my previous comment because I was wrong. ICH views risks as just events - a huge fundamental flaw. P2 is not a probability of an event. P1 is the probability of a cause event (situation). P is the probability of harm. P2 is the probability of a connecting mechanism that turns the cause P1 turning into harm P such that P1 * P2 = P. P2 is a risk mechanism defined by the connecting actions, processes, activities, systems etc. that turn the cause into the effect. P2 does not describe an event. Bayesian networks do not describe causal relationships, they describe correlation relationships and that is huge difference when it comes to analyzing risks because the mechanisms between the events is what matters.