Doubting the Value of Test Method Validation
When I first joined the medical device industry (more years ago than I care to admit), I faced the task of validating test methods used in design verification for a class III medical device. At the time, our team believed our test methods were solid but hard to prove through validation. Most of our test methods were destructive, complicating gage R&R (repeatability & reproducibility) evaluations since repeatability is confounded with part-to-part variability in those test methods. ?
We often required multiple rounds with added justifications and manipulated test samples to pass. Since we usually didn’t do much to improve the test along the way, I was left with the distinct feeling that test method validation was just a "check-the-box" exercise that had little to no real impact on quality.
A few years later, during a routine validation of our packaging test lab, one of the test methods we needed to validate was for measuring seal width (per EN868-5 which was applicable at the time). Because this was a very simple test using only a ruler and was non-destructive, I figured the validation would be a slam dunk. I mean, how could you use a ruler wrong? To my great surprise, we failed!
As I met with the responsible engineer, I was sure I’d uncover some error in the validation protocol or analysis. However, as we delved deeper, it turned out the validation correctly failed. The method called for the operator to choose multiple locations of the seal to measure. Some of our seals were quite long and could vary in width over the length. In the end, we modified the test method to provide instructions on how to select the locations to measure and subsequently passed the validation.
领英推荐
Two things about this test method and its failure stand out. First, it was very simple with no complex measuring systems or fixtures. Many organizations might accept a rationale documented in a memo instead of testing due to its simplicity. Second, it followed an accepted standard. Many places would say that their test methods do not need to be validated if they follow an accepted standard; I’ve received that specific answer from a dedicated packaging test house.
The problem with that answer, of course, is that the referenced standard is only part of the test method. The equipment, specific test instructions, and even the training provided to the operators need to be validated to ensure that the standard has been incorporated correctly and is fit for its intended use.
That experience significantly impacted how I viewed test method validation and provided an important lesson in technical humility before empirical evidence. Furthermore, that was not the last time a test method validation taught me that particular lesson; I had a similar experience validating ASTM F88 for seal strength testing, leaving no doubt in the value of test method validation.