The truth about testing

by Lonnie Palmer

True story: Jim McMahon taught English at a small, suburban high school in upstate New York where I was assistant principal. He was a rookie, so he taught a “school group,” a label for students with weak academic skills and a proclivity for social and school discipline problems.

Due to the teacher “seniority pecking order,” rookies like Jim McMahon were usually saddled with the most challenging students and expected to keep them from breaking the furniture while hopefully getting them to read a few books slightly more challenging than the easiest comic book.

As assistant principal, I saw more than one student from Jim’s school group class for typical behavioral issues: smoking in school, skipping class or ditching the entire day, fighting in the cafeteria, cursing out a teacher, etc.

After a year or two of teaching, Jim decided to buck the trend of weekly spelling tests, diagramming sentences, and rewriting one paragraph “essays” and have this “school group” perform a play, a move that was immediately viewed as heresy among the faculty room glitterati. Didn’t he know that these students had to take and pass the required Basic Competency Test? What was he thinking!?

"The Politics of Education K-12" is available on Amazon.

The sky is falling, the sky is falling

The Basic Competency Test (BCT) in the mid-1970s was the first of what would become a testing movement that continues today. And, as with today’s standardized state tests, BCT tests elicited panic in teachers who did not think the “school group” could pass the tests.

But the BCT tests were a joke. In fact, I overheard one math teacher say: “Give me two weeks, two pounds of bird seed and a grey squirrel with above average dexterity so I can teach him to hold the pencil, and I’ll get him to pass the BCT.”

And, of course, over 95 percent of the “school group” did pass with ease on the first try.

Given the fact that teachers taught their own homegrown less-than-challenging curriculum to “school groups” in New York State at the time it was little wonder the Basic Competency Test was designed with such low standards. Without knowing what was being taught (every teacher had his/her/their own curriculum), test developers had to allow for a low degree of difficulty.

Meanwhile, the college bound students took Regents exams with a more challenging and structured curriculum and very predictable tests administered in June, August and January.

Talk about a rebel

With little or no guidance and a groundswell of criticism, English teacher Jim McMahon threw caution to the wind and invested nearly two months of class time preparing for a public presentation of the Broadway play adaptation of Ken Kesey’s novel “One Flew Over the Cuckoo’s Nest,” which at the time was a hit movie starring Jack Nicholson. The theme of the movie – rebel with a bad attitude in a mental hospital strikes a blow for humanity – did not sit well with some of the faculty room crowd, and I was doubtful Mr. McMahon could pull this off with the motley crew assembled in his “school group” English class. Remember, these students were selected because they could not read and write well enough to be considered candidates for college. And they had to take the BCT test at the end of the year.

They also were not drama students, and this was an exceedingly difficult play for students to perform well.

One particularly sullen, long-haired boy who had been to my office several times would play the lead role of Randle McMurphy. While his personality fit the role, I couldn’t see him being successful with the discipline and teamwork required for the role. (At that time, I, like most everyone else in education, was convinced a segment of the school population was not going to succeed for whatever reason.)

A school-wide assembly during the last class period of the day gave Mr. McMahon’s class a chance to show their stuff to their high school peers with the performance of a single scene from the play as a teaser for the entire play which was to be performed later that evening.

"The Politics of Education K-12" is available on Amazon.

I was backstage in my assistant principal role trying to make sure things ran smoothly. Mr. McMahon’s students were obviously afraid of looking stupid in front of their peers, but it was also obvious they were well prepared. The audience started out a bit rowdy and more than a bit restless. But within three minutes the student actors had them paying close attention to every word, gesture and sigh. When the scene ended there was a roar of applause and appreciation from the students in the audience and smiles, hugs and slaps on the back for all the cast including “Mr. Sullen.”

I still remember the goose bumps I had when the applause erupted at the end of the scene. But I was still uncertain about how these students would do with the full play and an audience of adults including their parents whose previous contacts with the school came only when their children were in trouble.

Later that night the students put on an outstanding play with a minimum of slip-ups, flubbed lines and recoveries and some truly remarkable and effective student acting performances.

The audience was as appreciative as the students earlier in the day. The teaser performance appeared to have boosted student confidence, and the students in his class were beaming when they took their bows and still beaming a full week later when I saw them in the cafeteria or in the halls in our school. It was a total success. And the sky did not fall.

Even the weekly spelling test proponents in the faculty room quieted down briefly, grudgingly acknowledging that something meaningful had happened in our school.

Mr. McMahon left our school at the end of the year, and I never saw him again. But his impact on me as an educator was significant.

A few years later in my role as a high school principal, when I was reading books by Ted Sizer and John Goodlad and materials from the Coalition of Essential Schools, I realized what Jim McMahon’s class performed was technically called an “authentic assessment” of their English skills. An authentic assessment is a real test like an athletic contest or a concert performance by the band as contrasted with the inauthentic fill-in-the-bubble tests made by commercial test making companies.

As the psychometricians who develop the inauthentic tests will tell you, the bubble tests are statistically more reliable – the results can be repeated with different versions of the tests administered to the same test takers. Would Jim’s class have done as well, better, worse with a second play? Who knows?

Evaluating the academic success of individuals who participated in Jim’s play is cumbersome and tricky. Did the student who designed the sets really learn anything of value? Can any of these students read, write or speak better as a result of their work on this play? There are rubrics (complicated scoring systems) for evaluating essays, projects and even acting performances, but they are all approximations, and they all involve subjective analysis by the teacher.

Rubrics for something like the performance of “One Flew Over a Cuckoo’s Nest” are inherently more unreliable than the bubble sheet tests. Three highly trained teachers evaluating the same acting performance using the same rubric on the same student in the play would probably produce three different scores. On the other hand, rubrics and the bubble sheets cannot measure intangibles like student motivation and confidence or how students see themselves as learners and as people.

THE END

Next up: Authentic vs. Bubble Sheet Assessments

"The Politics of Education K-12" is available on Amazon.