How good is Google's NotebookLM at making a podcast?

Article: 2/4

This week’s challenge for NotebookLM is to create an enjoyable, accurate, and informative podcast using the source material. And what better source material than my book “Outstanding School Leadership”?

Having loaded my book “Outstanding School Leadership” into LM once more, I sat back, read a few emails - what an exciting life I lead - and waited for the podcast to be ready. A little while later, no, I don’t know how long, I got distracted by the emails and my desire to perfect the perfect flat white, and now I’m being distracted by the story of why I’m trying to perfect the perfect flat white. There is no hope.

What was I doing again? I remember. Evaluating LM’s ability to produce an enjoyable, informative, and accurate podcast based on the book “Outstanding School Leadership” by Peter Hughes; that’s me if you’ve forgotten or missed it the first two times. Why enjoyable, informative and accurate? A podcast must, first and foremost, be enjoyable. Who is going to listen to a boring, unenjoyable podcast? There is some merit in listening to an enjoyable podcast. But, for me, a podcast’s value comes from its ability to inform, which requires accuracy. Imagine listening to a podcast and having no idea which parts were authentic and which were falsehoods - it would send me crazy. I like the idea of using LM’s podcast for revision or as a way of jumping to the executive summary for the book, guide or policy. So enjoyable is great but accuracy is critical if the podcast is to fulfil its informative aim. Let’s face it: most people listen to podcasts to be informed.

Enjoyable

A BIG yes. The conversational style is perfect. LM inserts pauses, Umm.., Arr.., Err... and other speech patterns and imperfections to perfection. Have a listen. If I didn’t know better, I would swear to it that it was two real live Americans conversing about my book. The hosts of the podcast are even over enthusiastic like Americans. My books is exciting, of course, but is it that exciting? This got me thinking. What is the most exciting document I know? Aha! I’ve got it. Ofsted’s Inspection Handbook. A quick Google, and there it was in all its 100 plus pages of glory. A quick copy paste of the URL into LM, and it was off producing another podcast. Wow! I didn’t realise the Ofsted Inspection Handbook was such an exciting read. I’ve never heard two people so excited to talk about the Ofsted inspection handbook. That’s what it must be like?when Sir Michael Wilshaw and Sir Martin Oliver get together. A total Handbook geekfest. On a serious note, it was a great way to refresh myself on the handbook’s contents. It was a far cry from having to reread it.

Accurate

One issue I encountered was the podcast’s difficulty distinguishing my voice from others. It attributed the personal story of Matt Jones, OBE, who wrote the foreword, to me. Although his narrative about his mother’s efforts to send him to a better school was inspiring, it felt misplaced when presented as my own experience. The podcast’s handling of my suggestion that I might have dyslexia was contentious. While I merely hinted at the possibility, the podcast erroneously asserted it as a fact.

The discussion surrounding my involvement with Mossbourne Community Academy (MCA) was a little confusing. The podcast initially credited me with the academy’s turnaround, which was actually due to Sir Michael’s efforts. Although technically, no school was turned around, Mossbourne was a catalyst for transforming an entire London borough (Hackney). Sir Michael founded MCA after Hackney Down, a failing school on the same site, had closed years earlier. The podcast acknowledged that I joined MCA later and implemented a series of strategies to take MCA from start-up to mature school, to sponsor school and finally into a multi-academy trust, which was the case.

Informative

On a positive note, the podcast effectively highlighted key themes from my book, including collaboration, relationship-building, data-driven decision-making, and the importance of different leadership styles. These are crucial elements of effective leadership, and LM’s decision (can AI make decisions?) to emphasise leadership styles was appropriate.

One of the podcast’s strengths was its ability to bring the human element to the discussion. It certainly did this more effectively than NotebookLM’s written summary and perhaps even more so than in the book itself at times. This focus made the content more relatable, bringing to life critical insights through practical strategies and experiences.

The podcast also excelled at discussing the application of business principles and research to education, a topic I have always supported and championed through examples in the book. In the book, I made a big point of showing the messy side of leadership, talking about my countless mistakes and how I’d learnt from them. Once again, this strand of the conversation really brought the human side of leadership to life. It added authenticity to the discussion.

The repetition of specific themes and ideas, and at times whole phases, slightly diminishes the podcast’s overall impact. Maybe the intent was to reinforce key lessons, which would be understandable. Unfortunately, the repetition of exact phases felt more like I was being gaslit rather than a reminder of essential facts.

Summary?

NotebookLM podcasts are nothing if not enjoyable. It struggles with accuracy and has a habit of repeating itself like my grandfather does at Christmas. Inaccuracies like mistaking Matt Jones, OBE, for me or stating I was dyslexic when I only suggested I might be, are not mistakes one would expect from a well researched human presenter. That said, they didn’t impact my enjoyment or take from the informative nature of the podcast. The gaslighting, lack of internal consistency, and repetition did affect my enjoyment and distracted me from the podcast’s ability to inform. Trying to enjoy and be informed while being gaslit is a challenge. ?

I’ve listened to the podcast several times, and I’m left with a feeling of warmth each time. There is an overwhelming sense of the importance of relationships and the human touch in leadership—quite a juxtaposition for an AI podcast.?

I definitely wouldn’t rely on LM to be your sole source of truth for a book, guide, or policy. This is true of all AI at the moment. I would, however, use it to refresh one memory. It was the perfect refresher for the Ofsted Handbook. It also serves as a good way to find out about a book, guide, or policy - you will still need to do the reading - Google hasn’t solved that problem, if it is a problem, yet.

Grade

The exam has been written and sent off to the examiner for grading. It’s a grade 8 for enjoyment. I was, at times, totally absorbed. It would have been a perfect grade 9 but for the gaslighting. Accuracy is a grade 6. The information was mostly accurate, but the combination of incorrect statements and inaccuracy of attributing information to the correct source is a problem. The podcast was highly informative and scored a solid 7 in the area. Making the overall grade across the 3 papers (criteria) a 7. LM is definitely on its way to greatness but needs to put in the hours to memorise the facts and who said them.

Don’t take my word for it. Listen to the podcast, read the book and make up your own mind.

Podcast link (here).