Is there a better way to do exams?

“The exam board Pearson has launched a “major” consultation into the future of exams, following calls for GCSEs to be scrapped.” [https://schoolsweek.co.uk/pearson-launches-major-consultation-into-the-future-of-exams-amid-calls-for-end-to-gcses/]

A share of this article on LinkedIn caught my eye and fired up my imagination (Thank you again Maggie Slaughter for spotting it). If I could set up a brand new system to summatively assess the learning in schools in a way that was fair to a diverse range of learners while clearly signposting skills for employers and higher education institutes, how would I do it…

The big assumption is that traditional exams are “bad”. We’ve all seen that cartoon where a bunch of animals are being “interviewed” and the test is to climb a tree (somewhat disadvantaging the elephant and the penguin!) But what is it about exams that results in so many smart and talented young people ending up feeling like failures at such a critical point in their lives?

If you have a good memory for facts, enjoy the subject matter, have no reading difficulties, don’t suffer stress and anxiety when under pressure, can write for 90 minutes without getting cramp in your hand, have access to a quiet space for revision and a supportive environment then exams don’t really pose any great problem. But what about everyone else, dare I suggest, the majority?

When you have experienced teenagers completely losing it outside an exam room – panic attacks, nausea, hysterics or just simply running scared – you really start to question the wisdom of this form of assessment for all. This is why so many vocational teachers were up in arms when Gove et al decided to introduce their “changes to make qualifications more rigorous” back in 2014.[https://www.gov.uk/government/speeches/michael-gove-speaks-about-the-future-of-vocational-education] That saw the addition of “externally assessed” units (exams by any other name) in the new BTEC qualification specifications released in 2015.

But if exams are so bad, why are they so popular? It may come as a surprise to learn that until the 19th century exams were not used by British universities, although there is much writing about the use of exams in Imperial China long before then. Instead, oral assessment was used. It is suggested by some writers that up to the early 1800’s Oxford University had only four oral tests, the contents of which were allegedly well known! Entry into the great universities was more about who you knew (or your family background) rather than what you actually knew – maybe that’s where we get the old proverb from?

Oxford and Cambridge universities led the introduction of exams to assess the suitability of potential candidates, “In the 1860s, the presiding examiners travelled from Cambridge to exam centres, usually by train, wearing academic dress and carrying a locked box containing the question papers.” [https://www.cambridgeassessment.org.uk/news/how-have-school-exams-changed-over-the-past-150-years/] and that was the start of the Oxford and Cambridge exam boards.

If you take a cold hard look at it, a single exam paper taken in one sitting by a large group of candidates and then marked to a standard scheme is far more efficient than an individual “viva voce” exam per candidate. One obvious down side though, is that the exam questions cannot be dynamically tailored to the individual in the way a conversation can – hold that thought.

What about the vocational approach? The apprenticeship system dates back to the middle ages and probably earlier. Like the oral exam approach it was very much a 1:1 model but of course primarily skills based rather than academic – pretty much the division we still have between A-levels and level-3 vocational qualifications. But again it wasn’t until the mid nineteenth century that a formal technical and vocational education started to be offered in Britain. [https://libguides.ioe.ac.uk/c.php?g=482262&p=3298299]

Various attempts have been made to introduce new qualifications over the years, I clearly remember the introduction of GCSEs in the late eighties, being a part of the final cohort to take O-levels. Although at my school we were Guinea-pigs for the new GCSE Physics - for which I have both a CSE and GCE certificate!

More recently we have seen the “New Diploma” come and go (mid 2000’s) and now the introduction of the T-Levels, meanwhile no government has been brave enough to actually scrap A-levels, which we still hear referred to as “the gold standard”.

So what of the proposal to re-vamp GCSEs now? I have mixed feelings, while it offers an incredible opportunity it also worries me that these initiatives are so often led by politicians, high fliers from industry and leading academics who have, I suspect, never taught GCSE students. At this point I should also declare that I too have never taught a GCSE course, but I have taught A-level, BTEC, AVCE, Access to HE, HNC/D, CMI Diploma and Foundation Degree (first two years of honours degree level), so I do have experience of both vocational and “academic” routes.

Back to the question I posed to myself: how would I do it?

First of all I’d aim to create an assessment environment where candidates feel comfortable and are able to perform to the best of their ability. A silent sports hall, sat at a wobbly desk, hand writing for hours, is not what I’d call a comfortable environment.

How often do you see an teenager without a smart-phone glued to their hand? What proportion of their life do they spend in cyberspace? Probably more time than they spend outdoors! Therefore, it seems obvious that any new approach to assessment should involve a significant proportion of on-line activity.

This immediately excites me with the possibilities for changing the whole feel of an exam. For a start, on-line exams can be adaptive just as a 1:1 oral examination would be. With contemporary AI / machine learning behind it the system could also move away from strict multiple-choice answers too. Of course a well designed multiple-choice exam is just as challenging as a conventional exam (anyone else who has taken Microsoft MCSE exams or the PRINCE2:2009 certificate will probably agree).

However, we can move even further from the classic printed exam paper or questions on a screen, why not have a fully interactive, multi-media experience? Questions could be delivered by means of video clips for example and students who struggle with written work could answer verbally, using speech recognition technology. How’s that for diversity!

There is also the opportunity to test skills as well as knowledge here, for example if you are assessing someone’s coding skills let them write (or debug) some snippets of code, using a real IDE and then paste in the results.

At this point in my train of thought I was struck with a really crazy idea: why just assess a single subject at a time? How many people have a melt-down at the thought of a maths exam? (A shiver runs down my spine as I think back to my A-levels maths). But if maths is embedded into other exams – the sciences being obvious examples, but what about music theory for embedding logarithms? English is even easier to assess because every other exam is going to require a good level of written (or indeed spoken) English. Think about it from an employer’s perspective, do they need people who can tell coordinating conjunctions from modal auxiliary verbs or people who can write fluently, clearly, succinctly with appropriate style and tone?

So far so good for testing knowledge and understanding, but what about higher order cognitive skills or affective skills? I’m a great believer in engaging problem solving skills by means of realistic scenarios which allow my students to apply their knowledge and skills rather than regurgitating facts. The same proposed adaptive, AI based examination system should be able to cope with this, worst case scenario, some of the answers will need to be reviewed by a human marker. Or, here’s a crazy idea, introduce anonymous peer reviewing to assess even higher order cognitive skills! The system could easily identify two candidates working at the same level and on completion of the relevant section they could be asked to peer review each other’s answers (obviously preventing them from then going back and modifying their original submission).

What about practical subjects? How about using a webcam to capture evidence? In the future how about using virtual reality headsets to give them a completely immersive experience? After all, airline pilots train in a flight simulator and as I understand it their first flight in a real plane is with fare paying passengers! Just think of the amazing experiences students could have. I remember a few years ago when my colleague was teaching computer logic to a level-2 group (GCSE level) and a few of them ended up building logic circuits in Minecraft, for fun!

One concern around on-line exams has always been access to the Internet, but again I’ll take a radical stance on this. Let’s forget the Victorian concept of education, the Dickensian image of Mr Gradgrind drilling “facts, fact, facts” into the heads of his young charges! People worry that the information age has left us all void of facts and figures because everything is at our finger tips courtesy of our favourite search engine. But is that valid?

Once again I consider what employers would want. When I was the manager of an IT help-desk did I look for staff with eidetic memories who could reel off the exact sub-key in the Windows registry to immediately fix an issue with an Exchange server (okay, one of my team did do that once!) or did I look for staff who could communicate effectively with customers while quickly and effectively locating the relevant information on a vendor’s website and help them in solving their problem?

Yet again we could remove one of the greatest exam stresses i.e. “my mind will go blank and I won’t be able top recall a single fact!” I would suggest that unless someone intends to become a professional quiz show contestant, retaining facts and figures is not a vital employability skill, however, the ability to quickly find and cross-check a reliable source is far more useful (and in these days of fake news, an important life skill).

I realise of course that these ideas are probably just a bit too futuristic for our current generation of politicians to get their heads around (I heard an amazing quote of Amber Rudd’s, apparently she didn’t understand why someone kept bringing up encryption during a discussion about online security). But who knows, maybe we’ll be bold enough to take the first faltering steps down this new road and perhaps one day we’ll realise Douglas Adams’ concept of the Compu-teach (Hitch-hiker’s Guide to the Galaxy, Radio Series 2, “Fit the Eleventh”):

COMPU-TEACH: Good morning life-form.

PUPIL: Hi teach.

COMPU-TEACH: Are you sitting comfortably?

PUPIL: Yes.

COMPU-TEACH: Then stand up! Harsh Economic Truths, Class Seventeen. You are standing up?

PUPIL: Yes.

COMPU-TEACH: Good. Posit: you are living in an exciting, go-ahead civilisation. Where are you looking?

PUPIL: Up.

COMPU-TEACH: What do you see?

PUPIL: The open sky. The stars. An infinite horizon.

COMPU-TEACH: Correct! You may press the button.

PUPIL: Thank you.

FX: Button is pressed. A surge of energy

PUPIL: Wow! That feels nice.

[https://www.clivebanks.co.uk/THHGTTG/THHGTTGradio11.htm]