Psychometrics, what it is and why it matters

Measuring people

“At the office I worked in before that, my [organization] required all employees to take a personality test that divided us neatly into one of four quadrants: Doers, Creators, Deciders, or Thinkers, categories that would then define our roles in the department. Most of the others were Doers; there were a couple of Deciders, too. I was the only Thinker. My first thought was, I think I need to get out of here.”Pamela Paul

Photo by?JACQUELINE BRANDWAYN ?on?Unsplash

As humans we like to categorize things, make patterns and associations, and be able to label people and things in different ways. It helps us to make sense of the world quickly - and we hope - efficiently. And yet there is something poisonous about personality tests. We don’t want to be categorized, pigeon-holed or judged by others based on an assessment.

Even so, as a coach, there often comes a time where I and the client explore options for using assessments. Sometimes the measures are “prescribed” or recommended by the employer, organization, coaching company or someone else (which I admit will influence my willingness to work with them). And at other times, we get to explore and choose the appropriate assessment(s) to use for ourselves.

Photo by?Tengyart ?on?Unsplash

Personality and other assessments

Whether we are making the choice or the assessment is organizational standard practice, whenever I am discussing an assessment, one of the things I look at is psychometric validity and reliability. This matters to me greatly. I will explain why in a moment.

What is an assessment?

Anyone can create an assessment. A series of questions, purportedly measuring some aspect of cognition, emotion, affect, character and so on.?

I can ask you questions, and at the end of that questioning tell you “what the assessment says about you”. But that is just opinion. In this case my opinion. We have no way of telling if there is validity to the measure or that it is reliable - although I can make a pretty good guess.?

So why does that matter?

Well apart from the fact that measuring people based on opinion - and therefore underlying biases is potentially dangerous (think of the one drop rule in the US), it is also not helpful. It erroneously classifies people and contributes to false judgments about their ability and value to the organization.?

Unless an assessment is shown to (a) measure what it says it is measuring (validity) and (b) that if you measure the same thing with the same person at different times you get similar results - all other things being the same each time (reliability), the assessment is not really telling you anything valuable. Both of these attributes are essential to a useful assessment. And even if the assessment meets these criteria, it is still only one tool to use in helping you and your (potential) employee understand their level of “fit” in the organization.?

If the questions measure what they say they measure but the answers change just because it is a different day, time of day, the weather, or whatever, there is limited value because any measure is just a snapshot in time, not a tendency or trait. So it is unreasonable to draw conclusions, or suggest actions based on the results.?

If we get the same results every time, but we are not measuring what we claim we are measuring, that is not helpful either. The fact that Ruth always describes this shape as “square” means that the question, “what shape is this?” may be reliable - because it always gives the same result. But it has no validity in terms of measuring whether Ruth can do cartwheels (she can’t).

Checking the facts

We at ALLE LLC love assessments, and when a new one crosses our desks, we are curious, Yet we have learned that the first questions to be asked are “how do we know it works?” and “who validated that?”

Because it takes a great deal of time, money and effort to create, test and validate an assessment, there have been very few that we have come to depend on and use with confidence.

Photo by?Ahmed Zayan ?on?Unsplash

Another measure for us is what proportion of the people validating the results are independent of the creator of the assessment. We love things to be peer reviewed rather than the result of one group or organization’s collection of data because that truly answers the question, “is it repeatable?”?

There are some very popular assessments in the world that only the creators have tested and validated. And others that noone has validated - and yet others that have been called into question based on psychometric validity and reliability but that still get traction in organizations and schools. In the case of all data and testing being conducted by the interested party, the people leading that testing are often well-respected scientists and analysts and statisticians. Yet if the only people to validate a tool are those who have a vested interest in it being validated, well that is the fox watching the chickens - and we know what tends to happen with that arrangement!?

Our favorite assessments for you

Anytime we use an assessment with a client, you know we at ALLE LLC have done our homework. The same with third party training programs that we recommend.?

Some examples of ones that we use are:

The VIA Character Strengths Assessment.?

The basic assessment & ranking report are free.

This assessment has been around for a little less than 20 years and was based on the research by Dr Martin Seligman and Dr Christopher Peterson into what makes us our best. This research, started in the early 2000’s has continued with many studies around the world in multiple domains. While the assessment has been adjusted over the years to increase its effectiveness - validity and reliability - the results have been repeated independently in over 800 studies.?

In fact, any researcher can sign up to do research, get access to the algorithm and the items, and test it for themselves.?

The basic assessment is free, contains 72 items (reduced from the original cumbersome 240 and subsequent 120) and takes about 15 minutes to complete in your first language. At the end you get a ranking of your 24 character strengths from 1 - 24.?

This helps you to understand which character strengths you use most (in my case Appreciation, Bravery and Curiosity), and which are the hardest work to engage (in may case Self-regulation!)

To take your assessment and get your free results, you can use this link:

https://ALLE.pro.viasurvey.org ?

Measuring Mindfulness - the Five Facet Mindfulness Questionnaire:

Paper & pen assessment (see link below) is free.?

Another assessment I love as a mindfulness coach is the Face Facet Mindfulness test from Dr Ruth Baer (my love of this assessment does NOT stem from our shared first name.) This is an amalgamation of several separate assessments that measure the multiple facets of mindfulness - Observing, Describing, Acting with awareness, Non-reactivity, Non-judgment.?

The original assessment is a paper and pen assessment and can be accessed here for free. But consider combining it with your character strengths from VIA and get your $15 Mindfulness & Character strengths report which goes into more detail about the mindfulness facets and your top five character strengths.?

To purchase your report go here: https://www.viacharacter.org/reports/mindfulness-and-character-strengths

(Note the report is based on your free character strengths assessment plus and online version of the mindfulness assessment that has been created with permission from Dr Ruth Baer)

Measuring Affective Competencies - the Multidimensional Inventory of Personal Intelligences (MIPI) from Adaptimist Insights

There is currently no free assessment but a short version will be coming soon.?

A recent addition to the ALLE LLC assessment fold is the MIPI - Multidimensional Inventory of Personal Intelligence. This tool from Adaptamist.com was developed through research at the University of Trent and comes with its own handbook AND a book about the psychometrics!?

You can access free videos about the intelligences and the foundations of the assessment here:

Contact me at [email protected] to try out the assessment.?

Burnout Assessment Test (BAT)

Free Assessment available on paper and online

Since Herbert Freudenberger identified burnout in the 1970’s much of the research has been conducted by Christina Maslach alongside Professor Wilmar Schaufeli and others. Advancing the earlier work that created the MBI (Maslach Burnout Inventory), Schaufeli and his colleague Professor Hans De Witte, have developed an pen & paper assessment tool called the Burnout Assessment Test (BAT). This original assessment tool is available free of charge at the link below.

With the kind permission of the team, we have been able to create an online version of the tool that allows you to take the assessment and receive your results in a few minutes.?

To take the free online assessment go here: https://projectmotivator.com/burnout-assessment/ ?

All the information about psychometric validity and reliability as well as a concept of thresholds of burnout are discussed on the original website.?

NeuralPlan from NPMI

Condensed version available currently for $99 and Certification program available for $899.?

As a project manager, I have been looking for a training program that truly integrates the human side of projects into project management. After meeting Dr Josh Ramirez and Dr Jodi Bull Wilson in LA in 2019, I have followed their work with passion. With the addition of Dr Shari De Baets to the team, the NeuralPlan training has been launched that integrates theories of behavior and neuroscience into project management. The Institute for Neuro and Behavioral Project Management implores us to design around what we know about the human brain. We are not completely rational - as other disciplines such as Economics have already realized - but that does not mean we cannot develop good plans and forecasts and have successful projects.?

To find out more about the courses with NBPMI check out these links:

Condensed Course ($99 for limited time):

Certificate Course ($899 for limited time):

---------------------------------------------------------------------------------------------------------------

What this has to do with Authenticity, Burnout, Health & Wellness, and Resilience - the core of ALLE LLC's work?

Authenticity: using assessments can help us to really hone our understanding of who we are, put language around our beliefs and behaviors, and prime us to be able to choose who we are and how we act in different situations.

Burnout: assessments can help us to understand where we are today in terms of burnout, what tools we have easily at our disposal and give us ideas on how we can move forward as we conserve energy and recover from stress.

Health & Wellness: When we know our strengths, a little more about our personality and understand what human traits and tendencies may thwart our best intentions, we can apply that knowledge to make decisions consistent with who we want to be and how we want to act.?

Resilience: We cannot know how to adapt to the vagaries of life without knowing what tools are in our toolbox, understanding who we are, and appreciating how we are pre-wired to act. Team resilience comes from understanding the collective effects of human tendencies in multiple people. All the assessments, and tools above, support us in building the capacity to flex and adapt in the face of adversity.?

Look out for our small posts on BIG TOPICS - a series starting on Monday April 25th that will highlight nine cognitive moderators explained in the NPPQ and the 10 personal intelligences explored in the MIPI.?

See you soon!?

_______________________________________________________________________

Check out our upcoming Burnout Prevention Coaching Program

The program is online, starts in June (three choices of date, day and time) and comes complete with assessments, live group coaching, learning materials and more!