Defining Process Capability for True Position Tolerances.

Edited 12/2024. See new paragraph on calculating sigma.

Introduction

Close to twenty years ago, I received a validation report from a molder for cell phone housings. One of the critical dimensions whose process capability we asked them to find was the circular true position of a feature. I quickly saw that instead of reporting the true position tolerance of the feature, they reported the x and y coordinates. The tolerance they used for the measured values was half of the true position tolerance that I had requested. For those component measurements, they reported mean, standard deviation, and process capability index. Initially, I was upset that they had done it wrong. I wanted the mean, standard deviation, and process capability for the true position value, not the component dimensions. After a few days of exchanging emails and thinking about it, I saw that there was a problem with what I requested and couldn’t come up with a better way to do it. So, we went with what they had. As I recall, I assumed doing it that way was probably pretty close. As it turns out, it is off significantly.

The problem and potential solutions have been on my mind on and off since then. While searching for solutions didn’t turn up anything, it was hard to believe that no one had solved that problem. At one point, I even did a journal search and came up with a few leads but, until very recently, I never got copies of them (imagine shame emoji here). Another time I came to the realization that firearm accuracy and circular true position tolerances were very similar issues. Again, limited follow through on my part (imagine another shame emoji here). Finally, I am happy to report that I came up with a solution. While it seems that I merely reinvented the wheel, for some reason few people seem to be aware of this method, so this article will introduce the solution so that it can get broader exposure.

The Solution, Circular True Position

Let’s jump right into the solution. Background and discussion will follow to explain why this is the preferred approach. To find the capability of a process, you need the tolerance limits and the standard deviation. To find standard deviation, you need to find the process center. The following table compares step by step how the process center and standard deviations are found for standard tolerances and for circular true position tolerances and then how they are used to determine the process capability. The core of this method is to find the proper center so the proper standard deviation can be calculated. The two situations are presented side by side so the similarities and differences can be seen and understood. The similarities show that the two situations are in essence the same, with minor differences to accommodate the different situations. You may note that for clarity of the explanation, standard deviation, Cpk, and Ppk are expressed differently than normal but are in essence unchanged.

Figure 1 below helps visualize the process above. The green lines indicate the nominal center and the red lines indicate the center of the data. The outer circle is the circular true position tolerance zone and the inner circle shows what is 3σ away from the center of the data. The angled red line connects the center of the data to the tolerance limit. The bell curve shows a “side view” of the data set. In this example, the tolerance limit is a small distance beyond the 3σ limit, so one would expect a Cpk/Ppk of a little over 1. For this set, it is 1.19.

Why It Makes Sense: The Problem in Terms of Mean and Standard Deviation

The problem we are addressing is how to determine the process capability – Cp, Cpk, Pp, Ppk – of a circular true position tolerance. A standard tolerance gives the acceptable amount of deviation above and below the nominal value, typically in the form of ?1.50±.03. A true position tolerance uses basic dimensions, which have no tolerance by themselves, to locate a feature, such as a hole. A true position tolerance defines how far from nominal the center of the feature can deviate. Typically, but not always, it is a circular zone. See Fig 2.

The hole has its own tolerance, typically a plus/minus a value. Also, for simplicity, this article ignores the difference between short-term and long-term process capability. Briefly, the difference between those two is how the standard deviation is calculated. Standard deviation is a key part of process capability and its proper calculation is the root of the solution to this problem.

To find the process capability for a particular dimension, you need the tolerance limits and the standard deviation. To find the standard deviation, the process center is needed. Some see that the challenge of true position process capability is due to the fact that true position tolerance is a one-sided tolerance. This is true in how it is reported but not exactly true in actual physical representation.

Before calculating standard deviation, you normally find the arithmetic mean of the data. When using software, you don’t do this step, but the software does it. In the context of circular true position, let’s call it the center instead of the mean. Imagine plotting the measurements of a standard dimension along the number line. Assuming normal distribution, we would see many points close to the mean value and fewer further away. The data should roughly follow the bell curve:

Now, visualize plotting the results of a true position study. Because it is two dimensional, we need a grid instead of just a number line. It might look like this:

Instead of having a bell curve, you would have entire bell, or maybe a sombrero, like this:

In Fig 6, the circle represents the true position diameter for one arbitrary point. We are accustomed to finding the arithmetic mean to find the center of a group of measurements. For a circular true position tolerance, one might be tempted to find the mean value of true position values, which is by definition relative to the nominal center, green cross. Doing that would provide the mean radius of the data points about the nominal center. Mean radius might be interesting but is not used here in finding sigma. The center of the data is indicated by the red cross and that is what we use to find the standard deviation. As is expected, there are more points clustered in the center of the data.

Figure 7 is an example that helps illustrate the problem with using mean radius. Assume the circle represents the mean radius about the nominal center. Note the two red data points that are about the same distance from the mean radius circle. In a normal situation, values that are close to the mean contribute very little to the standard deviation. However, it is visually easy to tell that those data points are different distances from the center of the data and logically must make different contributions to increasing the standard deviation. This indicates that using mean radius to calculate standard deviation is not helpful. Also recall that two data sets can have the same mean but very different standard deviations so there is no relationship between the two.

There is something interesting to note. Let’s call R the distance from the nominal center, green cross, to the center of the data, red cross. (Fig 8 ) The bigger R becomes, the closer in value R and mean radius become. Conversely, and more importantly, for a process that is perfectly centered at nominal, R would be 0 while the mean radius is some positive value. (Fig 9) If you consider mean radius to be the center of the data, this shows how misleading using mean radius can be. When the center of your data is at nominal, mean radius, which will be a positive number, suggests that it isn’t.

Now let’s walk through how standard deviation is calculated for circular true position. This is the same deliberate process used as described in the table above. Start with finding the center of the data. Then for each data point, find the distance from the data point to the center, r. Next square each value and find their sum. After that, divide that sum by one less than the number of values in the data set[1]. Finally, take the square root of the result of that to get standard deviation. Here that is in equation form:

This explanation helps show the criticality of properly finding the mean or center. Without a proper center, the next calculations are inaccurate. A couple of other facts worth mentioning. With respect to calculating standard deviation, whether a point is above or below the mean does not change its impact to the result because the value is squared. However, with the mean or center, whether a point is above or below the mean dictates whether it increases or decreases the mean.

(added 12/2024)

There is an easier way to find the standard deviation. First, find the standard deviation of both the x and y components. Then take the square root of the sum of the squares, as if you were finding the "hypotenuse" of the x and y "legs". After I discovered this simplification, I found someone doing the same thing and called it the principal standard deviation. I can live with that name.

What’s Wrong with Using X and Y Coordinates?

With understanding how to properly calculate process capability for a true position, we can look at what is wrong with the method of just finding the process capability for the component measurements. In a word, it undercalculates the standard deviation. For each point, only part of the variation is captured. Look at the dataset of figure 10 below where the center of the data is indicated by the red cross and consider the indicated point. Along the y-axis, it is very close to the mean and thus changes the standard deviation very little. However, overall it is among the furthest from the center meaning it will make a large contribution to the standard deviation. It is true that the x variation is captured when the x component is analyzed but we can see that in the y standard deviation, some of the total deviation is missing. The same thing happens with when all the x values are considered, so both are going to be low. Therefore, both the x and y standard deviation calculations are low. A little playing in Excel suggests the process capability index is about 30% lower than it is using the preferred method.

The Solution, Linear True Position

Recall that true position tolerances can be both circular and linear. Above, we discussed the method for determining process capability for circular tolerances and now the linear case can be considered. Given that it is a linear tolerance, as you might have suspected, it can be treated pretty much like any other “standard” dimension and tolerance. The upper and lower limits are determined by the basic dimension locating the feature and the true position tolerance. The process center is simply the arithmetic mean of the data and the standard deviation is calculated per the typical method

?

Summary

This paper has described a method to determine process capability for features with circular true position tolerances. The primary effort of the method is to find the center of the process data and use it to find an accurate standard deviation. Other methods exist but have flaws which lead to inaccuracies. I have created a blog post if you want to go deeper into the other techniques. The method I independently developed and use for finding the standard deviation is described by a company called Esri and used in their software ArcGIS Pro. They call it “standard distance”.

You may have noticed that the images of the data points look like a shooting target after a few rounds of practice. Therefore, you would not be surprised to learn that they have also considered this problem. Very briefly, the preferred method seems to be finding the mean radius of each point about the center of the data, as opposed the nominal center, as used above. For a much more detailed explanation, see the Precision Rifle Blog, which in turn quotes other sources, so beware of going into a rabbit hole.

Feedback on this article is appreciated. Do you plan to use it? Would you buy software that set it up for you? Should I seek to submit this as a formal technical paper?

Thank you for reading.

Amusing Afterword

A few years after I discovered the true position problem, a quality engineer I worked with made a strange prediction. He said that someday there would be an Uschold process or Uschold corollary or something like that. Well, if you want to call this the Uschold method and fulfill his prediction, I won't complain. "Why should we," you ask because I even admit that I wasn't the first? Fair enough, but remember the New World continents are North and South America, not North and South Columbia. Finally, maybe Microsoft can add this unique standard deviation calculation and call it STDEVU. Food for thought...

[1] If you are finding the standard deviation of an entire population, divide by the number of samples, n. If your samples are part of a population, which is the case for determining process capability, divide by n-1. Worth noting, for the Excel STDEV and STDEVA functions, it uses the n-1 method. Use STDEVP for an entire population. As the sample size gets bigger, the difference approaches zero.