Guttman Errors: Additional Insight into Examinees
Guttman errors are a concept derived from the Guttman Scaling approach to evaluating assessments. There are a number of ways that they can be used. Meijer (1994) suggests an evaluation of Guttman errors as a way to flag aberrant response data, such as cheating or low motivation. He quantified this with two different indices, G and G*.
What is a Guttman error?
It occurs when an examinee answers an item incorrectly when we expect them to get it correct, or vice versa. Here, we describe the Goodenough methodology as laid out in Dunn-Rankin, Knezek, Wallace, & Zhang (2004). Goodenough is a researcher’s name, not a comment on the quality of the algorithm!
In Guttman scaling, we begin by taking the scored response matrix (0s and 1s for dichotomous items) and sorting both the columns and rows. Rows (persons) are sorted by observed score and columns (items) are sorted by observed difficulty. The following table is sorted in such a manner, and all the data fit the Guttman model perfectly: all 0s and 1s fall neatly on either side of the diagonal.