The results are in! - My...Robot Teacher? Will AI Replace Human Teachers? Part 3, The Student Response
Jennifer Harmon, PhD
Mixed Methods Research | Survey and Interview Evaluation | Associate Professor
The results are in!
Can AI replace Human Teachers?
Not if you ask my students.
While the short 5 question quizzes following the AI and instructor led videos yielded comparable results, students were nearly unanimous (93%) in preferring the instructor led video.
The students reported the preference for the instructor led video-
- Despite the instructor video covering a more difficult chapter
- Despite the instructor video being twice as long
- Despite the instructor’s Camtasia video editing skills being a work in progress!
I also asked my students reflection questions about their learning experience with the two videos. With their permission, I’m sharing some of the themes I saw reading through them.
When discussing what they liked about the AI video, students mentioned:
- The AI video avatar had even pacing and enunciation in it's automated speech (3)
- The AI video was a novel approach they had rarely experienced (3)
- The AI video simulation was higher in video quality (1)
When discussing what they didn’t like, students discussed:
- The AI video was boring (3), and not engaging (7)
- The AI video was not enjoyable to watch (3)
- The AI video was unnatural and distracting (10)
- Students appeared to feel most strongly about this point.
One student stated:
领英推荐
“The professor led is good, not as great as in person but much better than a glorified Siri reading a slide show to me as (I) idly listen with little care as there's no inflection or motivation to listen.”
Another student elaborated:
“I found the AI avatar to be very weird. While they tried to make the movements and habits similar to humans, the hand movements and eyes kept distracting me. I did not enjoy learning from the AI avatar. It was a little creepy and I had a hard time paying attention to what she was actually saying.”
One student further discussed that “I assume the information presented by the AI was provided by the teacher for it to present, so I don’t know that it is any better one way or the other is (if) all you are looking at is the information. I prefer personal interaction.”
This sentiment touched on the fact that there was not compatible AI with which to write the script for this source material so I did write the scripts for both videos. Also, several students mentioned preferring in person lectures (4), despite these lectures not being a part of the comparison for the class activity.
One student mentioned, “...it is a lot easier for me to pay attention and engage in a classroom rather than in this digital format.”
I realize there are several areas for potential bias in these preliminary results.
For one, the University of Wyoming prides itself on low instructor to student ratio (1:14) and average class sizes around 32 students. While this allows for more interaction between faculty and students, it also means that I ran this mini experiment on my smaller sized class, of which 14 students completed the activity.
Familiarity bias is another potential area of bias in these results. People tend to prefer ideas, places and people they are already familiar with. The students in my junior level course have already completed several courses with me at this point.
Finally, the videos were shown during an in-person session of an on-campus course. More than likely, lecture videos like these would be shown in an online course, where familiarity bias may have a diminished impact and video viewing by the students would not be as easily assured.
For a better apples to apples comparison, I would like to repeat a similar exercise, but would-
- Use an independent measures design where groups of students only see the AI video or the instructor led one
- Draft an AI video using an AI program to write the script and produce a video simulation
- Have the videos cover the same course topic and material
- Add an in person lecture condition to this comparison
It looks like, for now at least, UlearnAlone won’t be taking off any time soon.
Agribusiness Management Faculty at Penn State
1 年Nice little project and representation of its limitations, Jennifer. I will say it’s taken no experiment to determine that AI will never replace my ancient examples and student-specific jokes. Students would probably pick the avatar.
Humanitarian | Human Resources Professional | Enhancing a culture of care, work-life balance & respect for human dignity to boost productivity for non-profits | Advancing SDGs 8, 3, 4, 13, 10, 16 & 17 |
1 年Well thought out next steps to bring home the research. I was particularly drawn to the student feedback and though there may be some unintended bias, it was a good place to start. Looking forward to an extended research reach on this and would love to support the process to really find out if the sentiments will change in a different placement. Though AI is still advancing, perhaps in time it may be more relatable with less twitches.