A STORY ABOUT THE VALIDITY OF READING MEASURES

A Story About Validity

I was working in a special education resource room with students who were reading at beginning levels.?I administered an Informal Reading Inventory (IRI) before I began working with them as a diagnostic and to get a sense of their reading levels.?For the next several months, students' daily instruction included LEA, cloze activities, writing activities, word building activities, SPF for fluency, and reading practice.?For reading practice we used books that students liked.?These books had interesting topics with lots of pictures, color, they had appropriate sentence-to-picture ratios, and students could successfully read them.?Also, students were given choices about what books they wanted to read.?

Observations and simple progress monitoring measures indicated that we seemed to be making good progress.?Students were enjoying reading, reading more fluently, reading for meaning instead of sounding out words, and talking about the books they read.?In short, they had become readers.?After three months, I decided to quantify what I was seeing.?

Individually, I pulled students out of the environment we had used for reading and writing and took them into a larger room with fewer distractions.?For the IRI, I plopped down a page in front of them that consisted of text with no pictures or color.?Then I asked them to perform for me.?“Read this,” I said.

So, what is wrong with this picture??

I could tell by the non-verbals (something that would not show up on a standardized test) that something was up.?It was obvious that students did not want to be there.?I had pulled them from an environment in which they felt comfortable, where their friends and teacher were.?I asked them to do something that was not fun.?The text used in the IRI had no color, no pictures, and were of no interest to them.?They were just words printed on a black and white page.?Students were not able to use picture clues (like we had taught them to do) to help create meaning.?And I became another testing person, making them do things to show what they could not do.

Seeing the negative reactions from the first three students, I shut this process down.?The results were disastrous.?These students seemed to have become worse readers after all our hard work together.?

After a little thinking, I reassessed my assessment.?Instead of using the commercially prepared IRI’s, I decided to do miscue analyses with the books we had been using for reading in class for the last three months.?I recorded them reading these books in the classroom where they were taught.?I also created story-retelling charts based on these books to assess comprehension.?And, I gave them a choice of two books to read.?I asked, “Do you want to read _____ or ____?”?It was only a choice between two books, but it was a choice like they were used to.

My readers magically reappeared.?I saw an increase in meaningful miscues and self-corrections when compared with the IRI passages we had used for their diagnoses and baseline measures at the beginning of the year.?This told me that students were reading for meaning and improvements were being made in their metacognitive abilities.?The scores on the story retelling charts showed that the students were doing a better job of creating meaning with what they read.?Also, the reading fluency and prosody that I had observed during instructional sessions returned.?These improvements could have been due to (a) the intervention used, (b) the type of material used for assessment, (c) maturation and increased engagement with authentic literacy experiences, or (d) a combination of all.?

There are two lessons to take from this:?First, when assessment looks different from instruction and practice, you will often get results that under-represent students’ actual ability.?Removing a reading event from its meaningful context during assessment creates an artificial, non-reading event.?A valid assessment for reading should look, to the greatest extent possible, like real reading.?And second, student assessment of any kind always occurs within the sociocultural, emotional, and experiential context.?These variables must be considered when examining results.