Chatbot or Cheatbot? ChatGPT in the Exam Hall & Classroom

This is the second of three articles on what our initial reactions to #ChatGPT are telling us about the implications of this and similar #artificialintelligence #technology for educators and students.

In this piece, we explore specific concerns many have shared about its implications in the exam hall and classroom. If you missed the first, you can find it here:

CHATBOT OR CHEATBOT?

A common refrain in all the ChatGPT discussions we’ve seen is that it makes cheating easier, both in exams and coursework. To consider this critically, we’ll put to one side for the moment any bigger questions about the what, how and why of #education today. Instead we’ll start with the more practical issue of the likelihood of getting caught cheating using ChatGPT.

To stop cheating, we first need to be able to detect it

Cheating in exams has been around as long as exams themselves. When education technology consisted of chalk, ink and perhaps calculators, students cheated using simple, old-fashioned techniques like notes on shirt cuffs, or reading other candidates’ answers over their shoulders. And for these, there were simple, old-fashioned measures involving exam hall conditions, invigilation and so on.

The move to online exams nullified some anti-cheating measures, but others have replaced them. For example, professional qualification exams - such as cloud computing certification - use a 21st century version of enforcing exam hall conditions for online exams. Candidates must use a computer with a video camera, to confirm they don’t move or look away during the exam to find answers elsewhere, such as on another computer. They demonstrate they are alone with no aids present by a video sweep of the room before starting.

Of course, a concern with technology like ChatGPT is that it creates completely new ways to cheat in exams, not just modern versions of old ways. However, those voicing such concerns don’t always mention that technology also creates new ways to counter new forms of cheating. For example, one straightforward technology solution to new forms of cheating could be to temporarily monitor or control computers used to take online exams. This would allow examiners to detect or prevent use of other software by candidates during the exam - not just ChatGPT, but search engines, documents or communication tools.?

ChatGPT and coursework

Arguably, ChatGPT will have a greater impact on ways to cheat in coursework than exams. Judging by public reactions, the main concern is a supercharged reincarnation of the cheating headache the Internet initially created for educators. Teachers grappled then with how to spot work that was better than it should?have been,?for example copying, or submitting work produced by others. ChatGPT is creating a more sophisticated version of the same issue, and teachers may need more sophisticated approaches than previously. For this, technology will be their friend, albeit a poacher-turned-gamekeeper (or perhaps arsonist-turned-firefighter) kind of friend.

Anti-plagiarism tools like Turnitin appeared soon after Internet access and search engines became mainstream. In the same way, “anti-ChatGPT” tools are are already appearing, for example, GPTZero, which detects ChatGPT work by analysing text for new lexical characteristics like “burstiness” and “perplexity”. Turnitin’s owners are working on similar tools to spot computer-generated answers, and ChatGPT’s own product roadmap is said to include anti-fraud features.

As always, someone determined to cheat may find ways round any measures we use, regardless of technology. But it seems to me that sufficient measures will inevitably and rapidly evolve, and should allay most current fears of widespread ChatGPT-powered cheating.

CHATGPT IN TEACHING - OPPORTUNITY AND THREAT?

We’ll now shift focus to a more optimistic part of the early debate on ChatGPT in education: its role in teaching rather than assessment. Many - including some teachers - are of course worried about problems it may cause in the classroom. Those worries may well be valid, but let’s leave others to debate them elsewhere.

Our interest here is in the growing number of voices discussing how ChatGPT might be used as a teaching aid. Below are two examples of the many that captured my attention.

Using ChatGPT to demystify AI

AI has become ubiquitous, but many are still not aware of just how prevalent it is, or how to tell when AI is being used in daily life. When dealing with AI - including ChatGPT - few appreciate the basics of how it works, and therefore what it’s actually doing. As a result, it’s easy to assume it’s far more capable than it really is, for example that a computer using sophisticated, human-like language also uses equally sophisticated, human-like logic.

ChatGPT is a great example of advanced AI that isn’t as advanced as many will understandably assume. But we need to construct specific examples to show this, and examine them in a particular way to make its limitations clear. For example, comparing ChatGPT answers to differently worded forms of the same question can be illuminating, especially when context is important. Such examples could provide an engaging, useful reason for learning the concepts of how AI works in a non-technical way. It’s not hard to envisage such classroom discussions helping remove some of the aura of mystery that can shroud AI for the uninformed.

Understanding data, (mis)information and knowledge

Most people realise digital data is now an important element of most aspects of life. But the relationship between data and what we “know” is less clear, especially in a world of social media, fake news and of course increasingly powerful AI.

Some educators - such as in Scandinavia - see this as a teaching topic for schoolchildren, and I’ve seen mention of the opportunity to use ChatGPT to support this. This is another dimension of understanding how AI works, this time focusing on the content it provides, which is based on the data it feeds on.?

The heart of the issue is that what ChatGPT “knows” is limited to what it found on the Internet at a particular point in time. More specifically, it only found what its creators told it to look for and designed it to “understand”. This should trigger many questions for any ChatGPT user. Did it only consider verified sources, or does it also contain opinion-based data such as social media feeds? Would it use a single occurrence of a “fact” in its answers, or does it need multiple independent sources? What does it do with conflicting data about something?

Again, the huge interest in ChatGPT has made these questions more urgent for this particular technology, but we should be looking at AI in general with similar scrutiny. It may be too late for some adults to meaningfully consider such questions, but ChatGPT could be a powerful vehicle to ensure their children do.

Much of the early reaction to ChatGPT in education seems to have been about concerns, such as new ways to cheat in exams and coursework. But the combination of new technology and past experience suggests it’s reasonable to be cautiously optimistic about how we will deal with them. The answers aren’t here yet, and won’t be infallible when they arrive properly, but claims such as ChatGPT causing “the end of exams” seem both premature and unfounded.

For me, the more exciting part of the discussion so far is the opportunities ChatGPT creates to improve education. But as we’ll see in the final part of this three-article series, many of the issues and opportunities are inter-connected, and are part of a bigger discussion about education, intelligent technology and the future.

These articles follow discussions, comments and feedback involving Tim Burnett ? , Graham Hudson , the e-Assessment Association , Julia Cousens-Smith , Fran Carter , Hadrien Pouget , Owen Roberts , Rory McCorkle, PhD and Ahmed Zakaryya among others.

If you'd like to know when the 3rd article is out, or are interested other aspects of #aiethics, connect with me, Was Rahman - Ethical, Responsible AI , or follow the AI Prescience page.