ChatGPT generated a lesson plan for us and we taught it. Here's what we learned.

with Daniel Taylor-Griffiths

Generative AI is touted as a tool for easing the workload burden for teachers. One way it could do so is to assist with lesson planning. A recent study suggested that lesson planning is fast becoming one of the main purposes teachers are finding for generative AI. Much of the focus on generative AI in education has (understandably) been on assessment. There has been less emphasis on other implications.

We needed to refresh a lesson that hasn’t been working as well as we hoped, so we decided to put ChatGPT's lesson-planning abilities to the test.?

As a caveat, the lesson we planned for includes the development of capability in lesson planning as a core outcome. The course (subject/unit) the lesson formed part of is a postgraduate initial teacher education course in Queensland, Australia. The lesson in question is a large, two-hour interactive workshop session, on campus supported by extensive custom-made text-based and multimedia material online. We would be less confident in teaching a lesson produced by ChatGPT if it couldn't be used to open up a course-relevant conversation in the class about the lesson planning process and the possible role of AI in it. As such, there was no risk of negative effects on the progress of the student cohort. What we trialled here has direct relevance to their learning. We also have confidence in the course design around this specific workshop (thanks in no small part to Dr Emma Somogyi and Jennifer Nicholls ).?

Getting ChatGPT to produce the lesson plan

I (JL) produced the initial prompt. I don't claim to be an expert at prompting ChatGPT. I am still learning. I initially tried to give it as much detail as I could up front including the theoretical foundations I use to inform how I teach. I also clearly articulated that a key concept we aimed to discuss in the lesson is the notion of embodiment in learning. Here is the initial prompt I gave:?

The initial output was unusable. Despite the size and detail in the prompt, I did not give ChatGPT enough information about the context. In particular, the size of the class (just under 200 students in total) and the space (a large flat floor room) seemed to be important information to make the initial version more suitable. Interactive aspects of the initial lesson plan were not viable with such a large group. For example, the plan grossly underestimated how long it would take to hear back from groups and have a whole class discussion after a small group activity. After several iterations, we finally had a version that was viable and taught it as close to the plan as possible.

Preparing for the lesson

Preparing for the lesson was a little tricky. At first glance, the plan was overly structured. It also immediately struck us how poor the estimates were of how long every aspect of the lesson was gong to take. Having seen many lesson plans produced by students, colleagues, and others over many years, this is the kind of classic error that we often see, particularly the first time someone plans a lesson.?

ChatGPT did make quite a few useful suggestions. The plan included a clear set of lesson goals and outcomes. It also provided a handy list of required resources:?

We had to make some changes to the resources we used because we plan to delve into those kinds of resources at another point later in the course and couldn’t get bogged down on that aspect of the lesson. This was one of the few minor changes we had to make.?

Other elements of the lesson were fairly standard fare; an opening mini-lecture, an embedded video and two small group activities. The activities focused on a set of case studies and a short lesson planning task. ChatGPT didn’t include case studies in the plan so we prompted it to provide us with those separately. We incorporated a video we had previously used for the same lesson in previous years. The remainder of the resources we produced ourselves, including the slides, which were consistent with our usual resources.?

The first attempt at generating the case studies wasn’t specific enough. ChatGPT produced case studies about postgraduate education students and not about primary and secondary students, which is what our students need to consider. After being much more explicit about the parameters, the final prompt we used to generate the case studies looked like this:?

Again, the case studies were usable; not ideal but serviceable despite trying to give a lot of detail in a series of prompts. We integrated the case studies into the lesson.?

These all sound like standard lesson planning activities. These preparatory activities were not why we found this task difficult. I (JL) found myself constantly wanting to radically change the lesson because it did not suit the ways I usually like to teach. The ambitious timing was one issue but it was also apparent that the strategies ChatGPT produced would constrain the conceptual depth we could get to in the lesson. I found myself wondering if I would have been better off doing it myself from scratch.

Teaching the lesson

As was the case with the planning, teaching the lesson was also uncomfortable. We simply had less confidence in the plan generated by ChatGPT than we would have had in one we produced ourselves. Are we still smarter than a large language model? When it comes to understanding how we teach, who our students are, and how we can best help them achieve the outcomes, then it seems that we most certainly are.?

What we needed to do is to draw heavily on the range of tactics in our teaching toolkit to get a lesson that was not ideal to be as effective as our usual lessons. The faults in the strategy developed by ChatGPT became clearer in the teaching of the plan and we needed to work hard to dynamically adapt the lesson as it progressed. As an aside, I (JL) draw on Peter Goodyear’s work to separate pedagogical strategies and pedagogical tactics.?

The strategy was lacking and we needed to make up for the deficiencies with a range of tactics. For example, we needed to pause the lesson in various places and explain in greater depth what we were doing and why. We also needed to do more work to connect the case studies to the outcomes of the lesson. Our sense is that these adaptations seemed to have the desired effect and the lesson overall was better than the previous one I (JL) designed.?

Debriefing?

We did not obtain ethics approval to conduct a formal study and, therefore, do not have formal data collected from students. Our observations suggest that students were largely unaware that this lesson was designed in a radically different way from those in the course previously. The lesson did seem to feel more rigid due to the additional structure.?

We used the second half of the allocated time to have a discussion with students about what we had done, why we did it, and what our experience was. We also talked more broadly about the development of generative AI for education. We were mindful to ensure that the integrity of the constructive alignment in the course remained intact. A large portion of this discussion was devoted to lesson planning in preparation for an assessment task based on evidence-informed lesson design. Our impression is that this discussion was greatly valued but, again, we didn't collect any data to support that.?

Why was it difficult to teach a lesson generated by ChatGPT?

It seems to us that the plan produced by ChatGPT was generic to the point of being formulaic. Given what generative AI is and how it works, that is not particularly surprising. For people new to teaching or training, a standardised approach to a lesson can be effective. The level of detail ChatGPT provided could be useful for novice teachers and provide confidence that the lesson will be viable, even if not ideal but it does rely on some ability to give it the right prompts.?

As experienced teachers, we found the lesson difficult to work with. The plan was too constraining, the timing was all wrong and it did not fit with the teaching style we have developed. This outcome is likely not at all surprising for any experienced teacher or trainer who has had to work within a tightly bound syllabus or curriculum. Much of the art, creativity, and authenticity involved in high-quality teaching was stripped from the lesson.?

While there are undoubtedly many other teachers and trainers more experienced and effective than we could ever hope to be, we have been doing this long enough to have confidence in our own teaching style. ChatGPT didn’t give us a lesson plan to suit that style but that could be due to our limited capability for prompting it and not a limitation of the model. This issue raises the question of whether the relational and personal nature of teaching can be codified sufficiently to allow for an effective prompt to be written so that the resulting plan is more suitable. We are not sure.?We feel as though it's still much easier to do it ourselves.

What is needed to deliver effective lessons generated by AI?

Based on our experience here, it seems to us that the greatest possible benefit of generative AI for lesson planning is also the greatest risk. Lesson plans generated by ChatGPT and similar tools are likely to be most helpful for new teachers and trainers. However, those same new teachers and trainers don’t necessarily have the experience to know how to adapt lesson plans to maximise their effectiveness. Experienced teachers, on the other hand, are likely to get annoyed with the formulaic and generic plans it produces (at least based on our admittedly limited ability to write prompts) and will wonder if it would have been easier just to do the plan themselves. We certainly did.?

Our overall sense is that there are some interesting implications of generative AI for lesson planning. At this stage, it seems that teachers using generative AI to plan lessons need to have experience and/or be good at prompting (at least much better than us), preferably both. For new teachers and trainers, the lesson planning capabilities should be used with caution. For experienced teachers, the generic nature of the plans is likely to be annoying and stifle your style, creating an inauthentic learning experience. ChatGPT could, of course, be used to develop an idea that you could modify to suit your style and your students rather than just run the lesson as is like we did.

Generative AI unsurprisingly produces the same kind of generic lesson plans that have long been available via online repositories and earlier in books titled something like ‘the X ideas for teaching Y’. It is inevitable that generative AI will get better at tasks like lesson planning. However, our experience with a plan generated by ChatGPT as average prompters suggests to us that there will be an ongoing need for educational designers and impactful human teachers to do the bulk of lesson planning for the foreseeable future.??