CXC Faces Questions of Reliability and Validity

CXC is faced with two conundrums. The Reliability and the Validity of its assessments are under scrutiny. So why two conundrums?

Reliability Conundrum

Either CXC will hold to its guns and stick with the position that its assessments continue to be reliable: that is if the students who took the assessments this year had demonstrated a similar performance in any other year their scores would have been exactly the same. Or it will accept that there was an error in score calculation this year that caused these scores not to be reliable on the same measures as in previous years.

Validity Conundrum

Similarly CXC can stick to its guns and claim that these test scores can be used in all ways as the test scores have been used in previous years or accept that there may have been an error in scaling and thus, this year’s test scores cannot be validly used in all situations as previous year test scores.

Why do these situations pose conundrums? The only worth of any assessment lies it its reliability, that is the ability to reproduce identical scores for identical performances in a range of situations over extended periods of time, and its validity; that is the extent to which each use of the scores can be justified for the purpose that score is used.

So, either CXC holds on to the position that this year’s results are as reliable and valid as results in any previous year or it accepts that errors were made. The expected response is for the institution to hold with the position that its reliability and validity are un-impugned whether its members believe it or not. Why? Because if it accepts that errors were made under current leadership then current leadership will need to be replaced to reinstate the credibility and re-inspire confidence in the institution. 

The Fix

Let me add that with swift action this situation can be fixed. I am sure that CXC still has all the student performance artifacts and all the metrics derived from those artifacts. With new leadership stepping in and using impact data to realign all the performance standards and reissuing new results based on more appropriately aligned performance standards the institution can salvage its reputation. The longer this drags on the more difficult that will be.