SIGNIFICANT FIGURES AND STATISTICAL SIGNIFICANCE

When it comes to reporting data and setting acceptance limits, we are taught to report values with significant figures, and to set acceptance limits, which may be based on a statistical confidence level that predicts the boundaries of the population when the system is under control.

Many times, the lesson given to recognize significant figures is limited to looking for repeating zeros and assuming that these are position holding zeros.?By default, all other digits to the left are significant.?This is a poor presentation that does little to aid researchers on how to view real-life data that contain non-zero digits as background noise.?Furthermore, when a student reaches the level of statistics, the concept of significant figures usually drops out of the conversation, and isn’t seen again until the student enters the work force.?Analysts often rely upon instrument specifications to tell them the level of resolution that can be relied upon, but again, how was that level of resolution determined in the first place?

This discussion will also lead us to consider experimental design and the concepts of method and process validation.

Let’s start with the easier concept first.?Everyone recognizes the importance of significant figures, because it reports values in a way that reflects the “resolution” of your data.?I didn’t say “precision”, because precision is a validation term synonymous to “low variation”, which is not always the same concept, and I didn’t say “accuracy”, because accuracy is a validation term synonymous to “correctness or agreeability”.?“Resolution” is more appropriate, because it is synonymous to “the ability to distinguish small differences”.?While the concept of significant figures is easy to understand, and is understood to be important, application of this concept is not always intuitive.?It is easy enough to compare determined results against a specification to know how many digits to report.?But there is often a training gap in how the number of digits in the specification are chosen.

By definition, significant figures are those digits that are believed to be reliable and the first digit farthest to the right that is a best guess.?The first obstacle is that our numerical system is based on powers of 10, so we typically round our data to report the same number of significant digits that we see in our specification.?However, experimental resolution is not always based on powers of 10.?An example that I like to give is measuring the volume of a solution with a graduated cylinder.?The graduated markings make it easy enough to determine your volume to the nearest gradation mark.?But, it is common to be able to estimate the volume as being half way or even within quarters of the way between gradation marks.?In that case, the resolution of your method would be 1/4th of a gradation unit, which is also your best guess.?If each gradation mark represents 1 mL, then your resolution is 0.25?mL, and you may assume that you can report your result to two decimal places (e.g. 4.75 mL). ?But what if our resolution is 1/3rd of a gradation unit or 0.3333…, which gives you repeating decimals??We can’t easily convert to a base 10 number system without losing some information about the resolution of your value.?One way to retain the resolution is to maintain the fraction notation until you get to the final calculation, but this practice is not likely to become popular, because the data management is more complicated than merely entering a value into an electronic database, so most likely the resolution will be converted to decimal form.?Before converting your best guess to a value with decimals, remember that 0.1 represents a resolution of 1/10th.?If your best guess is larger than 1/10th, then you can’t justify reporting more than one decimal place.?So, a value of 4 ? mL should be reported as 4.8 mL and 2 1/3 mL should be reported as 2.3 mL.?Rounding is a reasonable approach for eliminating insignificant figures.?Using this understanding, we wouldn’t report two decimal places until our best guess has a resolution smaller than 1/10.?The place value for your best guess digit is calculated by:

Place Value for Last Significant Digit = 10^INT(LOG(resolution)).

The concept of significant figures is related to “precision”, because if a reading or test is performed repeatedly on the same stable sample over a short time frame and nothing other than the passage of time is expected to change, then we expect the result to be consistent.?Those digits in your data set that remain unchanged are significant figures, and the first digit farthest to the left that shows variation is your best guess digit.

Given the values: 3.75125, 3.76281, 3.75202, 3.74953, we can recognize that we have three significant figures (i.e. 3.75, 3.76. 3.75, 3.75).?Precision is usually measured by calculating the standard deviation (s or σ) or percent relative standard deviation (%RSD) also known as percent coefficient of variation (%CV). ?We may not be able to determine whether the insignificant digits are random or have a level of significance that fades as we move to the right, but we do know that they will show variation, and that the place holder value of the best guess digit will be reflected in the size of the standard deviation.?Since we know nothing about the degree of randomness in the insignificant digits, the standard deviation will have one significant digit, and if the placeholder value of the best guess digit is 10P, then the significant digit for the standard deviation will have a placeholder value of 10(P-1).?Before you say it, yes, this means that most of the time, the standard deviation is reported with several insignificant digits.?Perhaps one day, I will investigate to see if there is a way to uncover the potential existence of fading significance within this noise, but not today.

Many analytical teams have a practice of retaining all decimal places obtained from their instrument or test method and procedural calculations (i.e. “determined result”), and only round to significance after performing the final calculation (i.e. “reported result”).?This is contrary to the lesson most students get on the propagation of error.?However, I do not favor these rules either.?My reason is that the rules for maintaining significant figures throughout your calculations usually whittle away at the number of digits you have left to report.?They also assume the worst-case scenario, and do not consider that in your data, some errors may be self-correcting.?Sometimes, two wrongs can make it right.?Although to be candid, the most likely reason is that analysts have difficulty remembering these rules.?The practices that I see as a result of lost learning is why I am discussing significant figures here and other basic concepts in the past.?But, I also don’t favor these rules because they don’t always seem applicable to the real world.?Working in a GMP/GLP environment, it is not likely that I will ever need to combine results reported to a different number of digits.?Data is usually reported with a long list of conditional descriptions, so moving a test to another lab or facility or instrument usually requires some transfer evaluation.?The lesson that most students get on the rules for propagation of error are limited to basic arithmetic, and do not address things like fitting ELISA responses to a logistical curve fit.?Furthermore, although the concept of “accuracy” is critical to validation, proving true accuracy is like trying to prove the existence of God.?Accuracy is more appropriately thought of as “agreeability” with a reference value.?As long as the results are reproducible, and different values provide a scale for relative comparison, and the data remains within the context it is meant to be used, “true accuracy” is not critical, but that doesn’t mean that we shouldn’t make some effort to be as accurate as reasonably possible.?So please continue to read on.

Specifications tell the analyst two things.?As discussed above, the values in the specification reflect the number of significant figures to report.?The range defined by the specification sets the boundaries for acceptable values and rejected values, and that range is usually based on statistical significance.?We can’t just jump into statistical significance without first talking about experimental design.?Keeping true to the context in which the data may be applied is important.?

Validation or trending studies will often chart historical results to see where their values fall and will use a statistical model to predict a range where the population is expected to remain as long as the system or process remains under control.?A good validation study will capture all of the variables that would be encountered during normal operations.?The validation study will examine things like the accuracy and precision of the results, but too often, I have seen teams put their best man on the job or do whatever they can to demonstrate good accuracy and precision during the validation study.?This is not a wise approach, and is often why later they experience a higher rate of Out-of-Specification (OOS) results, than predicted from their data.?They set the bar too high by not being honest enough to show how different results may be between analysts, instruments, days, etc., which are all a part of normal day to day operations.?I don’t plan to move this discussion into one on validation, but do need to illustrate the difference between the data used to set a range for the specification and the data used to set the significant figures.

When collecting data to determine the resolution or ability to distinguish one measurement from another, we want to eliminate as many variables as possible.?Some validation studies already include “instrument precision”, but usually for the purpose of evaluating whether the product’s viscosity will shift the accuracy of an HPLC injection needle.?This data can also be used to determine the number of significant figures to use in your specification.?The significant figures give us the level at which we can distinguish one reading from another.?When we use statistical models to determine whether two sets of results are statistically different, we are including the variability between analysts, days, etc.?An analyst may prepare 3 replicate samples for testing and prepare them slightly different, so you get 3 close but different results.?The differences between those results are real, because your instrument and test method are sensitive enough to detect them.?However, you may not be able to distinguish 3 results from one analyst from 3 results from a second analyst, because they are not significantly different statistically.