Part 4: How reliable is the assignment to a certain personality type?

Series on typological personality models - Part 4

In part 3 of the series, I described in more detail why categorizing people into different personality types is problematic. The main point of this article was that typologies convey a false or at least highly distorted picture of personality - personality does not "work" in the way typological approaches assume. This is a problem of validity (does the content measure what is claimed?).

However, another major problem that has not yet been described relates to reliability (how reliable is the measurement?). There are different types of reliability, one of which is the so-called retest reliability. The idea behind this is simple: if the same characteristic is measured with the same procedure at two different points in time, the result should be the same or at least very similar. Provided, of course, that the characteristic being measured is stable. For example, if I want to measure something like "mood", it would be perfectly okay if two different results were obtained, because our mood can fluctuate greatly from week to week or even from day to day.

Results of typological procedures in part "as certain as a coin toss"

However, our personality does not fluctuate greatly, but is quite stable over time, as long-term studies show. Although it can be subject to changes over the course of a lifetime, we are talking about a period of several years or decades, and the fluctuations are generally not very drastic (Costa, McCrae & L?ckenhoff, 2019; Lucas & Donnellan, 2011). If we look at short periods of a few months or even just weeks, our personality does not change or changes only minimally. A good personality analysis must reflect this. It must come to very similar results at two (not very far apart) different measurement points in time. Under no circumstances should it paint a completely different picture of my personality in March than in April. Such a measurement would simply be worth nothing, like a set of scales that show 170 lb today and 260 lb tomorrow.

However, this is precisely the problem that often arises when working with typologies. For example, studies on the Myers-Briggs Type Indicator show that the probability of suddenly being assigned to a completely different personality type in a second measurement is 50 to 75%. Even if there are only a few weeks in between (Paul, 2010; Pittenger, 1993; Pittenger, 2005). In another study (not related to the MBTI), a team led by Prof. Dr. Matthias Ziegler from the Humboldt University of Berlin evaluated hundreds of thousands of data records and came to the conclusion that only 42% of people could be assigned to a personality type at all. This means that the majority of people could not be typed at all. To make matters worse, the probability that the assignment to a type was correct was only around 50% (Freudenstein, Strauch, Mussel & Ziegler, 2019). "The certainty of this assignment was about as likely as the outcome of a coin toss," says Ziegler (2019).

But why is this the case? There are various reasons for this, some of which would require a very detailed discussion of measurement technology. However, three hopefully easy-to-understand reasons will be mentioned here as examples.

Strong characteristics in independent personality traits can lead to a different "type"

One reason is that typologies sometimes make "either/or" decisions between personality traits that should actually be considered independent of each other because they have nothing to do with each other, see the example of "Thinking" (T) and "Feeling" (F) in the third part of this series. As Grant (2013) vividly describes, he was once labeled as "T" in the MBTI, but shortly afterwards he was suddenly labeled as "F". Had his personality changed so much in a short space of time? Of course not, it was probably simply because he scored high on both (which is not possible on the MBTI) and therefore sometimes scored "T" and sometimes "F" on repeated measurements. "When I scored as a thinker one time and a feeler one time, it's because I like both thinking and feeling. I should have separate scores for the two." (Grant, 2013).

Small changes or inaccuracies - big effects

Further reasons become clear if we remember the analogy with height and Karl and Tanja, who were already introduced in part 3 of this series. Karl is 5'6" tall, Tanja measures 5'8". Assuming that a typological method would draw the "boundary" between "tall" and "short" at 5'7", Karl would be "short" and Tanja "tall". It is immediately obvious that this is quite nonsensical and has already been discussed in Part 3.

But let's take the example a step further: we measure the two again four weeks later. What could happen?

1. It could be that Karl had grown one inch in that time. Would we be talking about a dramatic change here? Certainly not, after all the increase in height would be very small, hardly any observer would even notice it. A dimensional method would then also show this change quite unspectacularly. Time A = 5'6"; time B = 5'7". No big deal. But according to typological logic, Karl would suddenly be "tall". Although almost nothing has actually changed, Karl is assigned to a completely different type in the second measurement.
2. Perhaps we measured ourselves slightly when we first measured him. Maybe Karl isn't 5'6" tall at all, but 5'7". Such measurement errors are completely normal in personality measurement. Every personality method, no matter how good it is, always has a certain degree of imprecision, as we cannot measure such an elusive construct as personality with absolute precision. The consequence would be very similar to point 1. With a dimensional method, the result would be 5'6" on one occasion and 5'7" on another. We wouldn't know exactly which is correct but that's not too bad because the values are very close to each other and I get a very good idea of how tall Karl is, even if a small uncertainty remains. However, the typology would spit out "short" as a result once and "tall" once. So again, Karl ends up in a completely different pot (which automatically trigger completely different ideas in my head), even though nothing has actually happened apart from a small measurement inaccuracy.

The studies listed above, and the examples outlined here show that in many cases the result of a typological personality analysis should not be given any significance, as it is more or less random. This should always be taken into account. Be it in a selection interview, in coaching or in a team workshop - if the measure did not take place today but next week, the result could be completely different. How useful is this information then?

I would like to conclude by quoting Prof. Dr. Matthias Ziegler from Humboldt University, who says that it is not advisable to rely on type tests when making important decisions - for example in the area of human resources - because "there is a high probability that the typing is not correct."

In the next and final part of this series, I will discuss FAQs from the field.

I hope you stay tuned, best regards,

Martin Puppatz

+++++

Sources:

Costa Jr, P. T., McCrae, R. R., & L?ckenhoff, C. E. (2019). Personality across the life span. Annual Review of Psychology, 70, 423-448.

Freudenstein, J. P., Strauch, C., Mussel, P., & Ziegler, M. (2019). Four personality types may be neither robust nor exhaustive. Nature Human Behaviour, 3, 1045-1046.

Grant, A. (2013). Goodby to MBTI, the fad that won’t die. https://www.psychologytoday.com/us/blog/give-and-take/201309/goodbye-to-mbti-the-fad-that-wont-die

Humboldt-Universit?t Berlin. (2019). Vorsicht bei Typentests. https://www.hu-berlin.de/de/pr/nachrichten/archiv/september-2019/nr-19926-1

Lucas, R. E., & Donnellan, M. B. (2011). Personality development across the life span: longitudinal analyses with a national sample from Germany. Journal of Personality and Social Psychology, 101, 847-861.

Paul, A. M. (2010). The cult of personality testing: How personality tests are leading us to miseducate our children, mismanage our companies, and misunderstand ourselves. Simon and Schuster.

Pittenger, D. J. (1993). Measuring the MBTI… and coming up short. Journal of Career Planning and Employment, 54, 48-52.

Pittenger, D. J. (2005). Cautionary comments regarding the Myers-Briggs type indicator. Consulting Psychology Journal: Practice and Research, 57, 210-221.