Commentary - How to Teach and Work with ChatGPT

??????The release of OpenAI’s new large language model (LLM) ChatGPT in November 2022 has been the source of much worry about the future of education and assessment. Many, such as Lilian Edwards, Professor of Law, Innovation & Society at Newcastle University have foretold the death of the student essay. “I just think essay assessment is dead, really”, she writes, warning of the all-out replacement of critical thinking by way of LLMs.

???????????This kind of rhetoric has led to the banning of ChatGPT by some schools, including in New York City, the largest U.S. school district, and SciencesPo., one of France’s top universities.

???????????This is a mistake. Social media hype makes it too easy to overestimate challenges and underestimate opportunities. How should the University of Guelph community respond to this emerging technology? We suggest careful deliberation, grounded in a trusting attitude towards students, that recognizes not only the problems that ChatGPT presents, but its potential uses as well. Let’s first explain what, exactly, ChatGPT is.

???????????ChatGPT is a large language model (LLM). GPT stands for generative pre-trained transformer, a type of LLM that can generate human-like texts. What makes it “large”? The model has over 170 billion parameters that can be tweaked to fit the data it was trained on. ChatGPT was built on top of OpenAI’s GPT-3.5, which was trained on vast amounts of content from the web and other publicly available resources. It was then fine-tuned using supervised learning and reinforcement learning (RL). Human trainers generated and ranked many responses to any given input, and the resulting reward model was further refined with OpenAI’s own RL algorithm, proximal policy optimization (PPO).

???????????The result: ChatGPT – an LLM with the impressive ability to recognize language patterns and predict what its output should be given some input, with great speed and, sometimes, uncanny relation to what you’d expect from a human. ChatGPT can answer questions, hold conversations, write poems and essays, de-bug code, and much more.

???????????The worry: given ChatGPT’s abilities, students will use it to do their homework, complete essays, solve equations, and answer exam questions. Undoubtedly, some students will cheat using ChatGPT. They probably already have. However, our reaction to ChatGPT should not be panic and outrage. Students have been using mostly un-detectable cheating methods long before LLMs. Instructors should be ready and willing to explain how ChatGPT works and to highlight its potential uses and limitations.

???????????Yes, ChatGPT can produce student essays, but they often lack specificity, critical thinking, or insight. As ChatGPT will be the first to tell you, its “understanding of language is purely based on the patterns it has learned from the training data and it doesn’t have any understanding of the world, beliefs or moral values. Therefore, it can be argued that ChatGPT lacks any kind of understanding of meaning or knowledge.”

???????????Indeed, as the philosopher Eric Schliesser emphasizes, we shouldn’t confuse accurate prediction with knowledge. Moreoever, ChatGPT has been shown to be subject to ‘hallucinations’ (making stuff up), blatant mistakes in logic and reasoning, and doubling-down on falsities when challenged. These limitations should be made abundantly clear to students, and can be taught with eyes towards other important topics like misinformation, transparency, bias and trust.

???????????Herein lies the unique opportunity. ChatGPT may well represent the beginning of a sea-change in how many of us interact with artificial intelligence (AI). This means that education in digital literacy and the societal and ethical impacts of these technologies is of the utmost importance.

???????????First, we suggest taking the time to use and learn more about ChatGPT and other LLMs. Whether you teach in history, chemistry, or business, submit some of your class assignments to ChatGPT and assess it responses. It’s safe to say there will be things you are impressed with, but there will be limitations as well. Share these with students, and be open about the novelty, uncertainty, excitement, and worry about these tools. It’s all but certain that students already know about ChatGPT, and that if they’re not discussing it with instructors or TAs, they’re discussing it amongst themselves.

???????????We worry that anything less than open and frank conversations about ChatGPT with students will ultimately create an environment of suspicion and distrust which is not conducive to learning. Sure, tools are already being developed to detect AI generated texts, but we should not threaten students with the eventual ability to catch the use of ChatGPT, pretending in the meantime that it doesn’t exist. This adversarial approach not only disrespects students but undermines the possibility of collective inquiry into a potentially revolutionary technology. Does this mean that instructors may have to revamp some take-home assignments? Yes. Does it spell the end of critical thinking? Far from it.

???????????ChatGPT can be used to design novel assignments. It may be an excellent way for students to generate research ideas, to work on editing skills, to collaborate, and to become generally proficient at using these generative tools which are, for better or worse, going to be integrated in more of our day-to-day work.

???????????We take it as a given that students are going to use ChatGPT and so we suggest encouraging them to be transparent about how and why they have used it. Some of the ways they use it may be unacceptable, but some may be genuinely creative, helpful, and innovative. It will ultimately be up to the instructor to make this decision, but we should not close the curtain on ChatGPT before we’ve had the time to learn about its promises and pitfalls. Above all, we suggest working with ChatGPT, not against it.

???????????LLMs will change, their limitations will become clearer, and they will improve, all in ways which are hard to predict now. What we can predict is that they will not be going away any time soon. In our next newsletter, we’ll take a closer look at some specific strategies for teaching and assessment in the era of LLMs.