Frogs in a Pot: QA and the One-Size Approach to Disciplines, Programs, and Institutions in Post-sec
Jonathon Penny, Ph.D
Leader, Thinker, Innovator, Culture-maker, Problem-solver, Poet
From 2015. And even I have grown a bit too comfortable with the idea of formula-ized assessment in the intervening years, though I still insist on value-added deployment/execution. Worth thinking through once more (I think).
I'll summarize my concern as this: it's too easy, when standardizing anything, to overlook essential and productive differences. And when standardized anythings (testing, assessments, evaluations, mandates) are deployed--because one-size is easier for those doing or ordering the measuring--they eventually efface all those productive and essential differences, to our common detriment.
A pair of opinion pieces in the Chronicle, and a further meditation by a former BYU professor, on the value of the assessment craze.
I was caught up in some of that work (by assignment) over in the UAE. Of course, faculty were deeply skeptical, and of course administrators were frantically urgent about it all. I was somewhere in between.
As the department lead, influential across the faculty in the way I went about this, my effort focused on wresting meaningful control of assessment: 1) how can we add actual value to an exercise that is in the main simply a repackaging or remodeling of less formal activity? 2) what shifts or trends in student experience are these models responding to, and how can the process itself make us better in addressing student need?
Given that I have an aversion to wonkery of all kinds, and that I don't buy in wholesale to any theoretical model or ideology, I made benchmarking and outcomes exercises fruitful by offering meaningful analysis of 1) how our circumstances, and those of our students, resisted comparison with certain institutions and programs and invited it in other cases, and 2) whether or not "standardized" standards fit or addressed what our students dealt with in the main.
I agree to some extent with the second article that before the assessment craze, humanities program in particular were more likely to deal in amorphous, ineffable, vague "qualities," and that the demands of all of this assessment stuff have required us to think about proficiencies. And I think that kind of thinking can enrich both curriculum and pedagogy in ways that benefit students both intellectually and practically. We have needed in my discipline to better particularize not just what we want our students to know but what they ought to be able to do in ways that address real market expectations, and in ways that allow our students to articulate for both academic and non-academic audiences what it is they know and can do, and to think, more urgently than that, about translating and transposing that knowledge and those proficiencies into real-world situations.
But I'm skeptical of assessment offices, of the assessment industry, and of an institutional attempt to drive the conversation in any but the most general of ways. The specific work at the program and discipline levels can and should be shaped by a broader conversation, but it needs to be allowed to breathe and mean something on the frontline. A person who leads the discussion and supports good work is useful, but to have an administrative staff whose only object is to standardize terminology, templature, and terror across campus is incredibly destructive. And it doesn't need to be ongoing, changing with this or that sensibility, driving and justifying a new position, further padding an already bloated administrative structure. This kind of work is most effective when it is periodic and unconstrained: if it ain't broke, don't fix it; if it is, do.
All of this to say that, though I have experience with it that could land me a job as a director of assessment dot dot dot somewhere, I have just persuaded myself that I think this is a bad idea. I'd rather work within a faculty and keep doing the meaningful kind of appropriation I've done already, and really enrich and improve student experience and preparation for my trouble.
Image: unknown. Borrowed here from https://getit-fair.com/assessment/.
AI Training Data | NLP | Prompt Engineering | Multilingual Speech-to-Text Transcription | Chatbot | Conversational AI | Machine translation | Human in the loop AI integration
2 年Jonathon, thanks for sharing!