Evaluating Vial Washing Machine Performance Qualification Results: A Statistical Approach With A Case Study.

Introduction:

Expanding on the foundational concepts introduced in the previous LinkedIn article,"The Key to Quality and Consistency: A Comprehensive Guide to Vial Washing Machine Performance Qualification in Sterile Manufacturing, " " this piece delves deeper into the pivotal role of advanced statistical methods in evaluating Performance Qualification (PQ) data within the pharmaceutical industry's sterile manufacturing processes.

With an emphasis on fostering regulatory compliance and cultivating a culture of continuous process improvement, this article explores specialized techniques tailored for attribute data analysis. These techniques include hypothesis testing for proportions, exact binomial tests, the Chi-Square Goodness-of-Fit Test, and the application of Parts Per Million (PPM) reject rate, Process Z value and Sigma level as key performance indicators. A case study is presented to demonstrate how these methods can effectively extract valuable insights from PQ data, empowering validation officers to make data-driven decisions and ensure equipment compliance with regulatory rules and Good Manufacturing Practice (GMP) guidelines.

Through a systematic approach, the proposed methodology aims to validate machine performance with precision across multiple runs and various stages of the washing process. This comprehensive analysis involves strategic distribution of samples throughout different stages of the washing cycle and conducting multiple validation runs to guarantee reliability, consistency, and reproducibility.

By highlighting the significance of continuous improvement and adherence to stringent quality standards, this methodology seeks to enhance overall quality within the pharmaceutical industry, ultimately elevating the performance and efficiency of vial washing machines.

Optimizing Sample Size for Evaluating Vial Washing Machine Performance:

A critical aspect of validating the effectiveness of vial washing machines in sterile manufacturing is determining an appropriate sample size for performance evaluation. By employing statistical methods, validation team can calculate a suitable sample size to ensure reliable assessment, compliance with regulatory requirements, and promote continuous process improvement

Key considerations include:

Step 1-Define Parameters and Hypotheses:

In order to calculate the sample size, the following parameters and hypotheses must be defined:

? Estimated proportion of success (p): The expected proportion of vials passing the test. For high-performance equipment, the aim is for p = 1, indicating a 100% success rate. However, considering limitations, let's assume p = 0.99.

? Acceptable proportion of success (p?): The desired proportion of vials passing the test, set close to 1 due to the stringent requirements of sterile manufacturing. Let's assume p? = 0.995.

? Unacceptable proportion of success/ Hypothesized proportion of success (p?): The performance level below which the machine is deemed ineffective. Let's set p? = 0.98.

? Statistical power (β): The probability of correctly rejecting the null hypothesis when false. For β = 0.80, Z(β) = 0.842.

? Significance level (α): The probability of incorrectly rejecting the null hypothesis. For α = 0.05, Z(α) = 1.645.

With these parameters in place, the following hypotheses can be formulated:

o Null hypothesis (H?): The proportion of vials passing the test is > 0.98

o Alternative hypothesis (H?): The proportion of vials passing the test is < 0.98.

In the context of performance qualification of a vial washing machine used in sterile manufacturing, the defined hypotheses are more appropriate due to the following reasons:

A.???? Regulatory and Quality Assurance Perspective:

Sterile manufacturing is a highly regulated field with stringent quality requirements. Regulatory bodies expect manufacturers to demonstrate that their equipment consistently meets or exceeds established standards to ensure patient safety.

The null hypothesis (H?), stating that "The proportion of vials passing the test is > 0.98," ensures alignment with the principle of proving that the equipment consistently performs at or above this high standard, as expected by regulatory bodies.

B.???? Risk Management:

·??????? The defined hypotheses focus on a worst-case scenario approach. The alternative hypothesis (H?) indicates that any deviation below the 98% pass rate is a point of concern.

·??????? This conservative approach ensures that potential decreases in performance are detected, which is crucial for maintaining the high standards required in sterile manufacturing.

C.???? Statistical Validation:

·??????? The goal of statistical validation in this context is to ensure no evidence suggests that the equipment fails to meet the necessary performance criteria.

·??????? The defined hypotheses enable the validation team to confirm the equipment's reliability and robustness in meeting the ≥98% pass rate, providing a more stringent and protective measure.

?Step 2- Calculate Sample Size:

The formula for calculating the sample size for a one-sample proportion test is:

n = (Z(α) + Z(β))^2 p (1 - p) / (p? - p?)^2

Plugging in the defined values, pharmaceutical manufacturers can calculate a suitable sample size that accounts for the stringent requirements of sterile manufacturing, ensuring comprehensive testing and facilitating the assessment of stability and consistency within the vial washing process. Based on what mentioned above plugging in the values:

n = (1.645 + 0.842) ^2 0.99 (1 - 0.99) / (0.995 - 0.98) ^2

n ≈ 272.14

Rounding up, the sample size is approximately 273 vials. However, to ensure thorough testing across the entire washing process, the total sample size to 320 vials. This adjustment improves the reliability of the study and enables a more comprehensive assessment of the machine's performance.

To ensure comprehensive testing and account for variability in machine performance throughout the washing cycle, we propose dividing the total sample size into four parts of 80 vials. These parts will be distributed as follows:

• One part at the beginning of the normal batch washing
• Two parts in the middle of the normal batch washing
• One parts at the end of the normal batch washing

Furthermore, to ensure the reliability and reproducibility of the machine’s performance, it is recommended to repeat each validation run three times.

?Consequently, the total number of samples required for the study will be:

320?Vials/run×3?runs=960?Vials total

This comprehensive approach will help validate the machine's performance accurately across multiple runs and different stages of the washing process.

Step 3- Calculating Margin of Error (MOE):

Margin of error is a crucial concept that reflects the potential variation between the sample estimate and the true population value due to random sampling.

It can be calculated using the following formula

MOE= Z x square root of [ p? ?x (1 p?)/n]

where:

·??????? n is the sample size

·??????? Z-score is the value corresponding to the desired confidence level (e.g., 1.96 for a 95% confidence level)

·??????? p? is the hypothesized proportion of success

·??????? MOE is the margin of error

?by substituting the relevant values into the equation:

MOE = 1.96 x square root of [0.98 x( 1-0.98)/320] = ±1.534%

This means that there is a 95% chance that the true success rate of the vial washing machine is within the range of 98% ± ±1.534%. So, the actual success rate is likely to be between 96.46% and 99.53%.

Assessing Vial Washing Machine Performance Across Multiple Runs and Washing Cycle Parts:

To gain a comprehensive understanding of the vial washing machine's performance, it is crucial to evaluate its effectiveness at various stages of the washing cycle. Two key methods can be employed to achieve this: pooled analysis and the Chi-Square Goodness-of-Fit Test.

Pooled Analysis:

The pooled analysis approach involves combining data from all three runs for each part of the washing cycle and conducting the one-sample proportion test on the aggregated sample. This method provides an overall assessment of the machine's performance throughout each part of the washing cycle while also considering the variability between different runs.

The recommended sample sizes for each part in the pooled analysis are:

?? For the beginning part: n = 240 vials (80 vials per run x 3 runs)

? For the middle part: n = 480 vials (160 vials per run x 3 runs)

? For the end part: n = 240 vials (80 vials per run x 3 runs)

?By adopting the pooled analysis method with these sample sizes, a comprehensive evaluation of the vial washing machine's performance can be achieved across different parts of the washing cycle while accounting for the natural variability that occurs between multiple runs. This approach enables pharmaceutical manufacturers to gain a more accurate understanding of the machine's efficiency, identify potential areas for improvement, and ensure the quality of sterile pharmaceutical manufacturing processes.

?Chi-Square Goodness-of-Fit Test:

In addition to the pooled analysis, the Chi-Square Goodness-of-Fit Test can be employed to determine if there are significant differences between the three runs. This statistical test allows for the comparison of the observed frequencies in each run to the expected frequencies under the null hypothesis, which assumes that there are no differences between runs.

By performing this test, any significant variations between the runs can be identified, providing further insights into the machine's performance and consistency across multiple runs. The Chi-Square Goodness-of-Fit Test enables validation teams to make informed decisions, optimize processes, and uphold quality standards throughout the entire washing cycle.

Methodological Steps for Vial Washing Machine Performance Analysis: Data Collection, Presentation and Statistical Evaluation:

To perform a comprehensive statistical analysis of the vial washing machine's performance, following steps would be recommended;

?A. Data Collection:

1. Gather the test results for each part of the washing cycle (beginning, middle, and end) from the repeated runs (three times for each part).
2. Organize the data for each part in the tabular form as below format, ensuring the sample size (n) is equal to the collected 320 vials.

Here is an example of how the overall results for each part of the washing cycle can be presented, demonstrating the total number of passed and failed tests after each part has been repeated three times;

In order to enhance the understanding of presenting overall results for each part of the washing cycle, a case study has been prepared utilizing hypothetical results data for chemical contamination removal (Riboflavin testing). This case study will demonstrate how findings can be effectively presented and evaluated.

B. Binomial One-Sample Proportion Test of Pooled Attribute Data:

To perform a statistical analysis utilizing pooled attribute (pass/fail) data for each part of the vial washing cycle, a binomial one-sample proportion test will be employed. This test, based on the binomial distribution, serves to compare a single sample proportion to a target proportion. Its primary objective is to ascertain whether sufficient evidence exists to indicate that the true population proportion differs from the target proportion. Applicable for both one-tailed and two-tailed tests, however for our specific case, the one-tailed test is preferred over the two-tailed test due to the nature of the evaluation. The one-tailed test focuses on determining if the observed proportion is either significantly greater or less than the target proportion, which aligns with the goal of assessing whether the vial washing machine's performance meets or exceeds the desired quality standards. By using a one-tailed test, validation team can better evaluate the machine's ability to consistently maintain or surpass these standards, thereby ensuring product quality and regulatory compliance.

To execute the one-tailed binomial one-sample proportion test based on the provided case study data, follow the steps outlined below:

1.????? Calculate the pooled sample size for each part of the washing cycle:

o?? For the beginning part: n = 240 vials (80 vials per run x 3 runs)

o?? For the middle part: n = 480 vials (160 vials per run x 3 runs)

o?? For the end part: n = 240 vials (80 vials per run x 3 runs)

2.????? Determine the total number of passing vials for each part by summing the number of passing vials across the three runs:

o?? For the beginning part: x = (number of passing vials in run 1 + number of passing vials in run 2 + number of passing vials in run 3)

o?? For the middle part: x = (number of passing vials in run 1 + number of passing vials in run 2 + number of passing vials in run 3)

o?? For the end part: x = (number of passing vials in run 1 + number of passing vials in run 2 + number of passing vials in run 3)

3.????? Calculate the sample proportion (p?) of passing vials for each part:

Success probability (p?) represent the fraction of vials passing the test in the pooled data for each ????? part of the washing cycle.

o?? For the beginning part: p? = x / 240

o?? For the middle part: p? = x / 480

o?? For the end part: p? = x / 240

1.????? Calculate the Expected value or the expected number of successes under H?:

It represents the anticipated number of passing vials. To calculate expected number of successes each part by multiplying the sample size (n) with the hypothesized proportion (p? = 0.98):

o?? Beginning Part: np? = 240 * 0.98 = 235.2

o?? Middle Part: np? = 480 * 0.98 = 470.4

o?? End Part: np? = 240 * 0.98 = 235.2

?2.????? Evaluate the Null Hypothesis using Binomial Test:

The performance of the vial washing machine is analyzed through the conduct of a binomial one-sample proportion test, which compares the observed proportion of passing vials to a hypothesized threshold proportion. In this context, the p-value is computed using two possible methods based on the conditions met by the data to determine whether the null hypothesis for each part is accepted or rejected.

• ?Method I: Normal Approximation:

?When both the binomial sample size (n) and the product of sample size and probability of success (np?) are greater than or equal to 10 (np? ≥ 10 ), and similarly, the product of sample size and probability of failure is also greater than or equal to 10 (n(1-p?) ≥ 10), the binomial distribution can be approximated using a normal distribution.

In a one-tailed test, where the alternative hypothesis (H?) indicates that the proportion is greater than p?, a continuity correction of 0.5 is applied in the numerator.

Under these conditions, the following Z-score formula with continuity correction is employed to calculate the binomial test statistic, ensuring a more accurate and reliable analysis of PQ data;

?Z? = (x - np?) / √np?(1-p?)

Where;

x: The observed number of successes in the sample. This represents the data you've collected, such as the number of passing vials in each part of the washing cycle.

n: The sample size. This is the total number of trials or vials in each study consist of 3 runs

p?: The hypothesized proportion of success under the null hypothesis. In our case, it's 0.98, representing a 98% success rate or passing threshold.

np?: Expected value or the expected number of successes in the samples if the null hypothesis (H?) were true. It’s calculated by multiplying the sample size (n) by the hypothesized proportion (p).

(1-p?): The probability of failure (or the complementary probability of success). It's calculated as 1 minus the hypothesized proportion (p).

Square root of (np? (1-p?)): It indicates the standard deviation of the binomial distribution under the null hypothesis. It helps to understand how spread out or dispersion in the data points are from the expected value and represents the expected variability in the number of successes (X) if the null hypothesis were true.

The Z? score is used to quantify the difference between the actual and expected performance in terms of standard deviations, with a positive Z? indicating better-than-expected performance.

Upon calculating the Z? score, the standard normal distribution table can be consulted to approximate the p-value for the one-tailed test. This method incorporates the continuity correction to enhance the accuracy of the normal approximation for the binomial distribution.

For the given case study, with a sample size of 240 for the beginning and end parts, 480 for the middle part, and a hypothesized proportion of success (p?) of 0.98 and (1- p?) of 0.02 the conditions for normal approximation are calculated as follows:

As discuss earlier, in the analysis of the given data, it is important to note that the conditions for using normal approximation, specifically np? ≥ 10 and n(1-p?) ≥ 10, are not met for the beginning, middle, and end parts. This implies that the binomial distribution might not be well-approximated by a normal distribution in these cases.

Despite this, the calculation of Z-scores are proceeded with as an exploratory exercise to gain some preliminary insights into the differences between the observed and expected performances. This allowed for the examination of potential discrepancies and demonstration of the Z-score formula application. To further analyze these discrepancies, one-tailed p-values for the Z-scores can be calculated, providing the probability of obtaining a test statistic at least as extreme as the observed value, assuming the null hypothesis is true using suitable software like SPSS, R, or online resources such as socscistatistics . These p-values can be compared to a chosen significance level (p-value of 0.05) to determine whether to reject or fail to reject the null hypothesis.

Interpretation:

Although the p-value for the middle stage is slightly above the 0.05 significance level, it would be prudent to conduct further investigation to ensure optimal performance of the vial washing machine during this part. The p-values for beginning and end stages provide sufficient evidence to support (or fail to reject) the null hypothesis, suggesting that the proportion of vials passing the test is equal to or greater than 0.98 for middle and end part of the system.

While the overall performance of the vial washing machine appears to be consistent throughout the washing process, the borderline p-value for the middle part warrant a closer examination of the machine's performance during this stage. This may involve adjusting operational parameters or conducting additional testing under various conditions to gain a more comprehensive understanding of the machine's capabilities in removing contaminants.

Important note;? It is essential to interpret these results with caution, considering that the assumptions for normal approximation are not met. In this situation, employing the exact binomial test would provide more accurate inferences and enable a better understanding of the observed differences.

• Method II: Exact Binomial Test:

The use of exact binomial tests provides a robust method to evaluate the performance qualification of the vial washing machine, ensuring that the machine meets the stringent requirements necessary for sterile manufacturing environments particularly when the conditions for normal approximation are not met.

This method involves calculating binomial probabilities for all possible outcomes and summing those that are as likely as or less likely than the observed outcome under the null hypothesis. This comprehensive approach allows for a more accurate assessment of the probability distribution, especially in small sample sizes or when the success probability is extreme.

To fully understand the exact binomial test, it is essential to consider the different types of binomial probabilities that can be calculated:

?P(X = x): The probability of observing exactly x successes in n trials. Although not directly used in the performance evaluation, it helps in understanding the underlying binomial distribution.

1. P(X ≤ x): The cumulative probability of observing x or fewer successes in n trials. While not used in this specific case, it is relevant in applications focusing on a maximum allowable failure rate.
2. P(X < x): The probability of observing fewer than x successes in n trials. This can be used to assess worst-case scenarios or when considering a minimum required passing rate.
3. P(X ≥ x): The cumulative probability of observing x or more successes in n trials. This probability is crucial in performance qualification as it evaluates whether a system or machine meets a specified success rate or passing rate.

To simplify the process, common statistical software or online calculators can be used. Tools like SPSS, R, or online resources such as StatTrek can compute binomial probabilities without directly dealing with the formula.

In the context of performance qualification for a vial washing machine, the required values (n, x, and p?) for each part of the washing cycle can be input, and the necessary computations will be performed by the software or calculator. The primary focus is on whether the machine meets or exceeds a specified success rate. In this regard, the cumulative probability P(X ≥ x) is of particular interest, as it represents the probability of observing x or more successes in n trials. This metric is crucial because it directly assesses the likelihood that the machine's performance meets the desired passing rate. By comparing this probability to a predetermined significance level (α), such as 0.05, a determination can be made regarding the acceptability of the machine's performance or the need for further investigation or adjustments.

For the given case study the Cumulative probability P(X ≥ x) for each part of the washing cycle is summarized in the following table;

Interpretation:

To assess the performance of the vial washing machine, one-tailed binomial probabilities were employed, focusing on P(X ≥ x) to determine if the machine's performance meets the required success rate of 0.98. The results for each part of the machine were compared to a significance level (α=0.05) to draw conclusions.

1. Beginning part: Out of 240 vials, 238 passed the test, resulting in a 98% passing rate. The cumulative probability, P(X ≥ 238), is 0.13987. Comparing this p-value to the significance level (α=0.05), we fail to reject the null hypothesis that the passing rate is ≥ 0.98 for this part of the machine.
2. Middle part: Out of 480 vials, 470 passed the test, maintaining a 98% passing rate. The cumulative probability, P(X ≥ 470), is 0.63346. Comparing this p-value to the significance level (α=0.05), we again fail to reject the null hypothesis for this part of the machine.
3. End part: Out of 240 vials, 236 passed the test, also yielding a 98% passing rate. The cumulative probability, P(X ≥ 236), is 0.47478. Comparing this p-value to the significance level (α=0.05), we fail to reject the null hypothesis for this part as well.

The interpretation of these results suggests that, in each part of the vial washing cycle (beginning, middle, and end), there is not enough statistical evidence to reject the claim that the passing rate is ≥ 0.98. Based on these results, it can be concluded that the vial washing machine demonstrates high performance across all parts of the washing cycle given the specified threshold and significance level.

Chi-Square Goodness-of-Fit Test:

As previously mentioned, the Chi-Square Goodness-of-Fit Test can be employed to evaluate consistency across various aspects of the vial washing cycle. This test can be conducted in two parts as follows:

• Evaluation of consistency (the passing rates) across three parts (beginning, middle, and end), of the vial washing cycle within a single run
• Evaluation of consistency (the passing rates) across multiple runs for each part of the vial washing cycle

By conducting the Chi-Square Goodness-of-Fit Test for each part of the cycle within a single run and across multiple runs, any significant variations can be identified, providing valuable insights into the machine's performance and consistency. This approach enables validation teams to make informed decisions, optimize processes, and maintain quality standards throughout the entire washing cycle.

Evaluation of consistency (the passing rates) across three parts (beginning, middle, and end), of the vial washing cycle within a single run:

When conducting the Chi-Square Goodness-of-Fit Test, it is important to establish the hypotheses being tested. In this case, the hypotheses are as follows:

?o Null hypothesis (H?): There is no significant difference between the three parts of the vial washing cycle in terms of the proportion of passing and failing vials within a single run. In other words, the passing rates in each part of the vial washing cycle are equal.

o Alternative hypothesis (H?): There is a significant difference between the three parts of the vial washing cycle in terms of the proportion of passing and failing vials within a single run. This implies that the passing rates in each part of the vial washing cycle are not equal, suggesting that the quality of vial washing varies across different parts of the cycle.

?By comparing the calculated Chi-Square statistic and the critical value for a chosen significance level, validation teams can determine whether to reject the null hypothesis in favor of the alternative hypothesis or not. This information can then be used to make informed decisions about the vial washing process and ensure the highest quality standards are maintained.

Here's a step-by-step guide on how to do this:

Step 1: Calculate the expected frequencies:

For each run, calculate the expected number of passing and failing vials based on the binomial distribution parameters:

Probability of success (p)

Number of trials (n)

Step 2: Compute the Chi-Square statistic:

For each run, calculate the Chi-Square statistic using the observed and expected frequencies:

??2 = Σ [(Observed - Expected)^2 / Expected]

where ??2 is the Chi-Square statistic, and the summation is over all possible outcomes (passing and failing vials).

Step 3: Determine the degrees of freedom (df)

The degrees of freedom are used to determine the critical value in the Chi-square distribution table, which helps us evaluate whether the observed differences in the test results are statistically significant.

For a binomial distribution, the degrees of freedom can be calculated as:

?df = (Number of possible outcomes - 1) * (Number of runs - 1)

Since there are two possible outcomes (passing and failing vials) and three runs, df would be:

df = (2 - 1) (3 - 1) = 1 2 = 2

Step 4: Find the critical value:

To perform an analysis of the differences between runs in a binomial distribution, the following steps can be followed:

1. The Chi-Square table or an online calculator should be utilized to determine the critical value corresponding to the chosen significance level (alpha, typically 0.05) and the degrees of freedom computed in Step 3.
2. A comparison should be made between the Chi-Square statistic and the critical value.
3. If the calculated Chi-Square statistic (from Step 2) is found to be greater than the critical value (from Step 4), the null hypothesis can be rejected, leading to the conclusion that there is a significant difference between the three runs.

It is essential to note that this is a general method for analyzing differences between runs in a binomial distribution, and the specific calculations may vary depending on the data and assumptions involved.

?In the provided case study, the following data can be assumed for each part of the vial washing cycle:

• For the beginning part: a sample size (n) of 240, number of passing vials (x) equal to 227, and a target passing rate (p) of 0.98.
• For the middle part: a sample size (n) of 480, number of passing vials (x) equal to 462, and a target passing rate (p) of 0.98.
• For the end part: a sample size (n) of 240, number of passing vials (x) equal to 228, and a target passing rate (p) of 0.98.

With the given parameters, one can follow the steps outlined in the previous response to perform a Chi-Square Goodness-of-Fit Test. First, let's calculate the expected frequencies for each part:

Using a Chi-Square table as blow,for a significance level of 0.05 and df = 2. the critical value is approximately 5.991.

Interpretation:

Since the calculated Chi-Square statistic (1.83) is less than the critical value (5.991).

This indicates that there is insufficient evidence to reject the null hypothesis, suggesting that there is no significant difference between the three parts of the vial washing cycle in terms of the proportion of passing and failing vials.

Evaluation of consistency (the passing rates) across multiple runs for each part of the vial washing cycle:

As previously mentioned, the Chi-Square Goodness-of-Fit Test can be employed to evaluate consistency across various aspects of the vial washing cycle. By doing so, valuable insights can be gained into the consistency of the machine's performance across multiple runs. In this regard the following hypothesis might be assumed;

o Null hypothesis (H?): There is no significant difference in the passing rates for each part of the vial washing cycle (beginning, middle, and end) across the three runs. In other words, the passing rates are consistent across multiple runs for each part of the cycle.

o Alternative hypothesis (H?): There is a significant difference in the passing rates for at least one part of the vial washing cycle (beginning, middle, or end) across the three runs. This implies that the passing rates are not consistent across multiple runs for at least one part of the cycle, suggesting that the machine's performance varies between runs.

Here's a step-by-step guide on how to do this:

?Step 1: Calculate overall passing rates:

• Determine the passing rate for each run using the formula:

?Σ(passed Vials) / (ΣPooled sample size)

• Calculate the overall passing rate for each part of the cycle by combining the passing rates from all three runs. This will give you the expected values (E) for each part of the cycle.

Step 2: ??Calculate Chi-Square statistic:

• For each run, calculate the Chi-Square statistic for each part of the cycle: Subtract the expected value (E) from the observed value (x) for each part of the cycle in each run. Square the difference for each part of the cycle in each run. Divide each squared difference by the expected value (E) for each part of the cycle in each run.
• Sum the results from the previous step across all three runs for each part of the cycle.

Step 3: Calculate degree of freedom and critical valve at desired significance level:

• Determine the degrees of freedom (df) using the formula:

df = (number of categories - 1) * (number of runs - 1)

Here, "number of categories" refers to the distinct parts of the cycle (beginning, middle, and end), and "number of runs" is the total number of runs conducted (e.g., 3);

df = (3 - 1) (3 - 1) = 2 2 = 4

• Choose the desired significance level (e.g., 0.05).
• Use a Chi-Square distribution table or an online calculator to find the critical value corresponding to the chosen significance level and degrees of freedom.

·??????? Using a Chi-Square table as shown above, for a significance level of 0.05 and df = 4. the critical value is approximately 9.488.

?Step 4:?? Compare with critical value:

• Compare the calculated Chi-Square statistic with the critical value (4 df, 0.05 significance level) = 9.488 for each part of the cycle (beginning, middle, and end). If the Chi-Square statistic is greater than the critical value, reject the null hypothesis, indicating a significant difference in passing rates across multiple runs for that specific part of the cycle. If the Chi-Square statistic is less than or equal to the critical value, fail to reject the null hypothesis, suggesting that the passing rates are consistent across multiple runs for that part of the cycle.

With the given parameters and mentioned procedure, the following table can be generated to summarize the Chi-Square statistic for each part of washing machine:

Interpretation:

Based on the Chi-Square statistics and the critical value:

• Beginning: 0.0085 (less than 9.488) - Fail to reject the null hypothesis.
• Middle: 0.03 (less than 9.488) - Fail to reject the null hypothesis.
• End: 0.0086 (less than 9.488) - Fail to reject the null hypothesis.

Since all three Chi-Square statistics are less than the critical value, we fail to reject the null hypothesis for each part of the vial washing cycle. This indicates ?there are no significant differences in passing rates for each part of the vial washing cycle across the three runs. This suggests that the vial washing machine's performance is consistent throughout the process.

Evaluating Process Capability:

Process capability is a pivotal concept in industries that necessitate consistent and high-quality output, such as the pharmaceutical industry. It assesses how effectively a process or system can produce products that adhere to specified limits and customer requirements. Process capability analysis enables organizations to identify potential issues, optimize processes, and ensure product quality, safety, and efficacy.

In regulated industries like pharmaceuticals, evaluating process capability is essential for several reasons:

1. Regulatory compliance: Pharmaceutical companies must adhere to stringent quality standards, such as Good Manufacturing Practices (GMP) and regulatory guidelines set by agencies like the U.S. Food and Drug Administration (FDA) and the European Medicines Agency (EMA).
2. Patient safety: Consistent and capable processes ensure the production of safe and effective medications, which is critical for patient safety and well-being.
3. Efficiency and cost reduction: Identifying process issues and enhancing process capability can lead to improved efficiency, reduced waste, and lower production costs.

Process Capability for Binomial Distribution:

?When a process follows a binomial distribution, its capability can be evaluated using various metrics such as Parts Per Million (PPM) or Parts Per Billion (PPB) reject rates, Process Z value and sigma level. These metrics provide valuable insights into the proportion of defective parts in relation to the total number of parts produced and the overall process capability.

1. ?Identifying trends: Analyzing data from the three runs using these metrics can help uncover any trends or patterns in the reject rate, thereby offering a better understanding of the machine's performance over time.
2. Bench-marking: Calculating these metrics can provide a benchmark for the current performance level, which can be useful for comparison purposes as more data is collected and further analysis of the process is conducted.
3. Identifying improvement opportunities: High values of PPM, low Process Z values or Sigma levels in any of the runs may indicate areas for improvement or potential issues in the washing process that need to be addressed to enhance the overall process capability and maintain product quality

Important note; While calculating these metrics can offer valuable insights, it is essential to continue monitoring and analyzing the process to ensure its stability and consistency over time. Regularly updating these calculations and using other Statistical Process Control (SPC) tools, such as control charts, will provide a more comprehensive understanding of the process capability and enable you to make data-driven decisions for improvement.

Process Capability Analysis for a Vial Washing Machine:

Evaluating the process capability of a vial washing machine is essential for pharmaceutical companies to ensure consistent and high-quality washing results. As mentioned above one method to assess the performance of such a machine is by calculating the PPM (Parts Per Million) defect rate and using it to determine the Process Z value ?and Sigma level for each run.

Here's a step-by-step guide :

1. Calculate the PPM reject rate for each run: Determine the total number of defective vials and the total number of vials inspected in each run. Then, use the formula below to find the reject rate in parts per million (PPM): PPM = (total reject numbers in each run × 1,000,000) / total inspected
2. Analyze the PPM reject rate: Assess the machine's performance by examining the PPM values for each run. Higher PPM values indicate a greater number of defects and may suggest areas for improvement in the washing process.

4.??Determine the Process Z value based on the PPM reject rate: Using the PPM values, find the corresponding Process Z value in the reference table below. The Process Z value is a measure of process capability that considers both process centering and variation. Higher Process Z values indicate better process performance.

5.??Establish the Sigma level for each run: Convert the Process Z value into a Sigma level using the same reference table. Sigma level represents the number of standard deviations a process can fit within the specified tolerance limits. Higher Sigma levels signify better process performance.

With the provided parameters and the described procedure, the following table can be generated to summarize the PPM rates for each run and the overall performance across all runs;

Furthermore performing Binomial Capability Analysis on the given data using Minitab?? 20.3 provided the following invaluable results;

Interpretation:

1. Run One: In the first run, a total of 320 vials were inspected, with 6 rejects. This corresponds to a Parts Per Million (PPM) rate of 18,750, indicating that for every million vials, 18,750 defects could be expected. The corresponding Process Z value ranges between 3.5 and 4.0, which refers to a Sigma level between 4.2 and 4.6. These values suggest that the process is marginally capable but has room for improvement.
2. Run two; In the second run, a total of 320 vials were inspected, with 4 rejects, resulting in a lower Parts Per Million (PPM) rate of 12,500. The corresponding Process Z value remains between 3.5 and 4.0, which refers to a Sigma level between 4.2 and 4.6. These values suggest that the process capability level is similar to that observed in Run One
3. Run Three; This run demonstrated the best performance among the three. Out of 320 inspected vials, only 1 was rejected, resulting in a significantly lower Parts Per Million (PPM) rate of 3,125. The corresponding Process Z value improved to between 4.0 and 4.5, which refers to a Sigma level between 4.6 and 5.0. These values suggest that the process capability improved compared to the previous two runs.
4. The overall PPM defective rate suggests that 11,458 vials out of 1,000,000 are expected to be rejected, corresponding to a 1.15% defective rate. The 95% confidence interval for the defective rate ranges from 0.57% to 2.04%. This interval might be too wide for your application, and gathering more data could increase the precision of the estimate.
5. The Process Z value of 2.3 suggests that the process performance is adequate but might have room for improvement. A higher Z value typically corresponds to a more capable process.
6. Based on the process capability analysis results, the process appears to be stable and moderately capable. However, it's important to note that only 3 subgroups were evaluated in this analysis. For a more robust capability analysis, it is generally recommended to collect data from at least 25 subgroups over a long enough period to account for various sources of process variation.

7.?With the current analysis, the proportion of defective items is stable, and no points are out of control. The P chart and the Cumulative %Defective plot confirm the stability of defects in the process. The assumptions for the capability analysis appear to be satisfied However, it's important to note that only 3 subgroups were evaluated in this analysis. It is important to note that for a more robust capability analysis, it is generally recommended to collect data over a long enough period to account for various sources of process variation.This will help in identifying improvement opportunities and ensuring a more reliable process capability analysis.

Conclusion:

?This article highlighted the application of advanced statistical methods for analyzing vial washing machine performance in sterile manufacturing, building on previous article; "The Key to Quality and Consistency: A Comprehensive Guide to Vial Washing Machine Performance Qualification in Sterile Manufacturing, ". It showcased how hypothesis testing, exact binomial tests, the Chi-Square Goodness-of-Fit Test, and and the application of Binomial Capability could effectively derive valuable insights from PQ data, supporting data-driven decision-making and regulatory compliance.

A case study was systematically analyzed as a simple demonstration, revealing that the vial washing machine's performance remained consistent within and between runs throughout the washing cycle. Each method provided valuable insights into the machine's performance:

• Method I (Normal Approximation): While the machine's performance met the required success rate for the middle and end stages, the borderline p-value for the beginning stage indicated a need for further investigation. These results should be interpreted cautiously, as the assumptions for normal approximation were not entirely met, making the Exact Binomial Test a more reliable method for inference.
• Method II (Exact Binomial Test): The Exact Binomial Test results showed insufficient evidence to reject the null hypothesis, suggesting that the vial washing machine consistently demonstrated high performance across all parts of the washing cycle, meeting the required passing rate.
• Method III (Chi-Square Goodness-of-Fit Test): The Chi-Square statistic obtained indicated no significant difference between the three parts of the vial washing cycle in terms of the proportion of passing and failing vials, further supporting the machine's consistent performance across the washing process.
• Binomial Capability index: The Binomial Capability analysis indicated a 1.15% defective rate, suggesting that the process performance was adequate but may have room for improvement. The third run displayed the highest capability level, which could serve as a benchmark for future optimization efforts. It is essential to consider that these results were based on a limited data set of 3 subgroups, and more data should be collected for a comprehensive capability analysis

Integrating advanced statistical techniques in PQ data analysis empowers validation teams to ensure equipment compliance with GMP rules and codes while promoting a culture of continuous process improvement within their organizations. This holistic approach ultimately contributes to achieving the overarching goals of the pharmaceutical industry: maintaining high product quality and ensuring patient safety.