Using AI for Audiobooks

When I planned the 2024 themes for my monthly newsletter at the end of 2023, I thought this month's newsletter was going to be more generally about how authors can use AI:

• Ideation
• Research
• Editing
• Headlines
• Title and subtitle
• Proofreading
• Marketing funnels
• Audiobook narration
• Social media content
• Designing webinars
• Prepare launches

However, as I worked on the February newsletter about what characterises bestseller books, it became clear that a shift has happened (not quite unexpectedly).

?? I realised that I should start recommending that authors seriously consider publishing their audiobook first, or at least give it a much higher priority in their book strategy.

The reality is many authors either skip publishing the audio version altogether or do so as an afterthought.

In case you missed the article you can find it here:

The Art of Nonfiction Success: Decoding the secrets behind best-selling books and what sets them apart from the competition!

In the research, we looked at the overall bestseller list and four of the biggest nonfiction categories (Self-help, Health, fitness & dieting, and Business and Money).

?? Among other things, we scrolled down 40 books in each category and found that:

• 20 books occupied 54 of the 160 places across the 4 categories - meaning they rank in multiple categories
• 28 of the 54 places (52%) were audiobooks!
• In 20 occurrences, the audio version ranked higher than its print/ebook cousins! (same book)

???? You should publish all formats - and probably consider making the audio version the one you promote the hardest and launch first!

Many authors find it troublesome to create an audio version of their book. And true! It used to be a bit of a hassle and took quite a lot of the author's time unless they compromised and used a professional (expensive) narrator.

?? The good news: It has become a lot easier!!

This article helps you decide if and how to use AI to narrate your audiobook and includes a list of available narration tools.

I also share how you distribute it - and the trap you should look out for! ??

As a part of writing this article, I used multiple AI tools to narrate my latest book 'How to become a nonfiction author: Tips to writing and self publishing a book without losing your f*cking mind'.

?? The audiobook will soon be available! ????

?? Disclaimer:

Using tools mentioned in this article is at your own discretion and Publishing Rebel is not liable for any damage, financial or otherwise that might occur.

What is AI narration?

AI narration is the process of text-to-speech. Based on the text version of your book, it can very quickly, and for a fraction of what it costs to rent a studio, transform your written content into audio files.

The AI narration tools will break up the text into the required number of files and ensure they qualify for retailers to approve them and include them in their catalogue.

?? Do not skip the rest of this article as we dive into significant limitations on which retailers will approve which books!

Audio narration comes in four forms:

• Human narration using your voice

• Similar to podcasting; recording yourself reading your book using a high-quality microphone

• Human narration using a hired voice

• A professional voice-over artist is recorded reading your book in a professional studio

• AI narration using pre-defined voices

• You use a narration platform to transform your text into speech. The platform generates audio files using synthesized voices. Multiple platforms offer a wide range of voices and languages.

• AI narration using your cloned voice

• You record or upload audio files to a narration platform. Based on this content a cloned voice is created. This voice can read any text - oftentimes in any language, depending on the platform. The platform generates audio files based on the cloned voice and the text version of your book.

Services and tools available cover all these methods - either as done-for-you (DFY) or do-it-yourself (DIY). This article focuses on DIY tools available to independent publishers.

Creating audio files using pre-defined voices (the quick fix)

Pre-defined voices are usually based on your ebook. Some platforms allow using PDFs, Google Docs, or Word. If your ebook is already available on the chosen platform, publishing an audio version can happen in minutes!

Some suggested tools for AI narration using pre-defined voices are:

• Google Play

• For a limited time, there's no charge to create, publish, and download your?AI-narrated audiobooks. Publishers can sell audiobooks auto-narrated on Google Play on any retail platform that allows them but the audiobook must also be for sale on Google Play Books.

• Apple Books digital narration

• In a few steps, turn your ebook into an audiobook. The digital voices are created and optimized for specific genres though and Apple is currently only accepting books categorized as fiction, romance, mystery and thriller, or science fiction and fantasy.?
• Nonfiction is only offered through preferred partners, Draft2Digital, Publishdrive or Ingram Core Source. None of them seem to offer a self-narration service and it doesn't seem like it's even AI narration yet.
• Apple will work on your audiobook to have it "ready in approximately two months". Seriously?!?

• Designrr

• A simple tool anyone can figure out. You can choose from a (small) selection of standard voices and make small adjustments to them. The advantage of Designrr is primarily that you get an more-in-one tool to create ebooks and flipbooks besides AI narration. Audio is not their core competence and there are no advanced features.

• Virtual Voice (KDP Amazon)

• Currently, KDP is only available to beta users and only four standard voices are offered. If your ebook is already on KDP, this is a quick solution.

• Fliki.ai

• Although, you can create audios from text, I did not find this a relevant tool for audiobooks at this point. But it has some fun features! You can easily create shorter videos and audios from text, ie. audiograms to promote your podcast or audiobook. But for audiobooks, its a hard pass from me.

This way of making an audiobook available to your audience really is easy. Pick a voice. Edit if allowed. Publish. This is something you can do in less than an hour!

An obvious downside to these tools is that readers will not be listening to YOUR voice. Thus, you are missing out on one of the strongest forces of audiobooks: being inserted directly into the ears of your audience - the experience doesn't get more intimate than that!

Creating audio files using your (cloned) voice (the branded experience)

Using your voice ensures an uninterrupted brand experience. As you prompt your audience to jump from the book to the next step in your book funnel, it is powerful when they feel they already know you and your voice!

So let's take a quick look at the process of using AI narration tools to create your audiobook in your voice.

Before you start narrating, have the following ready:

• Finished manuscript (word, ePub, or PDF)
• Audio files or videos like podcast episodes to create your cloned voice
• Audiobook ISBN

You should also read through your manuscript to identify places where extra words are needed, ie. introducing lists, illustrations, and opening/closing credits which differs from the print version.

When you are done preparing, there are five steps to creating an audiobook using an AI narration tool (after choosing your tool, of course, which we will get to in a minute):

1. Clone your voice
2. Fine-tune your voice
3. Autonarrate your book
4. Proof and edit files
5. Export files in retailer-approved quality

Some suggested tools for AI narration using a cloned voice (your voice) are:

• Authorvoices.ai

• Offers both pre-defined voices and cloned voices, as well as done-for-you services. The price depends on duration; 1 hour (around 9,300 words) is $30, decreasing to $22,91 per hour at 10 hours) + $50 for cloning your voice. Register for a free trial.
• 24 pre-defined voices are offered with a good mix of gender, US/UK/African English, age, and ethnicity - but that's not what we are looking for here. It is a sign of a stronger language model though.
• Cons: Not very intuitive or user-friendly. I experienced lots of tech glitches and had to try everything multiple times. In fact, I had so many issues, I gave up. Maybe it will work better in your browser? ????♀?
• For some reason, I don't understand, I also found Authors Voice from a website called Audie.ai. Perhaps they are developing a new platform ????♀?

• Elevenlabs.io

• You'll need the Pro plan for $99/month to create a lengthy audiobook (the price jumps from 2 to 10 hours and 2 won't be enough). The Pro plan is also what you need to meet quality requirements from audiobook distributors (192 kbps bitrate) and to be able to create a professional voice.
• You can create voices of different levels of quality. The Professional Voice Cloning (PVC) will give you the best result. Once you've created your cloned voice, expect the process of fine-tuning to take quite a while - currently up to 4 weeks is what you should expect! In my case, it took 25 hours and was worth the wait!
• There is an Instant Voice Cloning (IVC) feature available too. It allows you to create a voice within minutes and based on only a tiny amount of audio material. I did not find the quality quite at the level I want for my audiobook and I even uploaded three chapters from my previous audiobook as input for the cloning. You could easily hear it was me though so I you don't have a lot of audio material, this is for sure an option!

• Descript

• I am a subscriber to Descript on the Creator plan ($12/month). I have used this for many purposes including transcribing and editing audio files and videos). I also use it to create show notes for the Publishing Rebel Podcast.
• The overdubbing feature (transforming written text-to-speech) works ok when you use it to add missing or mispronounced words to a larger audio file that already exists. As I tested using this feature to create audio files from scratch, I was less than happy with the result. It sounds robotic. The Creator plan is not designed for this use and many of the words will be read as "jibber" and "jabber" to force you to upgrade to the Pro plan. Whether a Pro plan will also deliver a less robotic sound is not mentioned. The result was so far from what I am looking to achieve so I didn't test the Pro plan.
• Note: Describt is not super user-friendly for your first transcription if you are not tech-savvy.
• If you want to listen to what the quality is like if you record old-fashioned style reading your book and use Descript for editing, get the audio version of my previous book, NextGen Author. That is exactly how I created that audiobook in a weekend!

• Play.ht

• You can create an instant cloned voice for free and test up to 12,500 characters for free. In my test, it is certainly my voice - but not quite my accent. They are currently working on a multilingual language model which might help non-native English speakers to better results.
• There is a more precise model available (High Fidelity Clone). You'll need the Unlimited plan to access that ($79,20/month). Like for Descript, the results I got with the free version was so far from what I want to achieve that I chose not to test the High Fidelity Clone. If you are a US native speaker, I do think this can be a great choice for you though!

• Listnr.ai

• Although the free version is not great when it comes to tonality and flow of the voice, this tool has some great features like detailed settings for pauses and a pronunciation library. I was inclined to test the paid version but it is missing features specifically for audiobooks, especially dividing the text into the required chapters and exporting the right formats. For serious authors, I just don't think it does the job.

• Podcastle

• Again, the free version is so robotic that I chose not to test the paid version. The free version only allows you to test pre-defined voices - to get access to Revoice, you need to upgrade. It's only $23.99 though so maybe it's worth trying. The free version is VERY simple - and probably too simple - but it does give a hint about user-friendliness in the paid version too. I think its a tool to re-test later. Audiobooks aren't their main focus but audio is!

• Lovo.ai (Genny)

• Ok, this will sound weird but I really liked this tool despite not liking my cloned voice at all. However, Genny is super user-friendly and offers a free trial on the cloned voice (which should be a standard but isn't) and some great features like speed, a pronunciation library, an ocean of voices, and the possibility to share directly to your social media. So I decided to sign up for the Basic at $29/month. This is just enough to test. If your book is more than 5 hours long, you'll need the Pro+ plan which is $149/month. But you would only need it for that month. It would also include stock images and videos for visual content creation.
• Even though Genny is apparently built on Elevenlabs technology, I did not get an improved voice when upgrading. It can still only be built on 5MB audio files which means the pool of data to learn my voice from is too small. For this reason, I cancelled my subscription immediately. I will keep an eye on this tool though!
• At the time of writing this, you will not only get a lower monthly fee by signing up for a yearly plan, you'll also get 50% off the first year.

So what is my conclusion on the best tool for AI narration?

At this point (April 2024), it is really only Elevenlabs.io that I find relevant to AI narrating audiobooks but a remarkable amount of tools have popped up in the market over the past 18 months.

I recommend one of these three options:

1?? Quick and dirty

Get excellent quality in minutes - in somebody else's voice

2?? Semi-quick and semi-dirty

Get good quality that sounds like you quickly - but spend enough time editing to weed out what you don't like. Elevenlabs.io is the preferred choice at this point. I haven't tested if the files will be approved yet but with the setting available I trust it will be fine.

3?? Slow and clean

Get into your closet (perfect studio!) and record your audiobook using a good-quality microphone (I use a Yeti). If you mess up, just pause and read the section again with the desired energy. Upload the files to Descript to be transcribed. Remove filler words using the feature available for that. Then edit repetitions out in the text and export the files at 192 kbps - 44.100 - 24 LUFS. Send them to an audio editor that you find on Upwork - someone who is experienced in creating audiobook files.

???? If you use option 3, make sure your recording quality is technically good enough. Send a sample to the editor before you record and have him/her optimise it so you can listen and see if you are happy with it. Also, practice reading your entire script multiple times before recording. Record one chapter at a time and remember to record opening and closing credits.

??IMPORTANT!!

Before heading into your closet to record or signing up for an AI narration tool, you NEED to read the next part of this article to ensure you have access to distribution!

?? Distribution of AI-narrated audiobooks

Some of the biggest players when it comes to audiobook distribution are not at the forefront of AI, to say the least:

• Author's Republic - only allows for human voice
• ACX (Amazon company) - only allows for human voice
• Bookwire (“WAY – We Audiobook You”) - human voice + Apple partner
• Findaway Voices by Spotify - only Google Play (which you will remember do not offer AI narration in your voice but only pre-defined voices).

???? Only Kobo allows you to submit any AI-narrated audiobook, no matter where and how it was created ????

Kobo only asks that you indicate the use of AI by listing your narrator contributor as?Read by?"Synthesized Voice,"?"Female Synthesized Voice,"?or "Male Synthesized Voice".??

The problem is that Kobo offers a limited distribution that doesn't include any of the largest retailers.

Kobo will automatically distribute your audiobooks to Kobo.com and the following partner sites:

• Booktopia (AU):?https://www.booktopia.com.au/?
• Librería Porrúa (MX):?https://porrua.mx/?
• Gandhi (MX):?https://www.gandhi.com.mx/?
• La Feltrinelli (IT):?https://www.lafeltrinelli.it/?
• Mondadori (IT):?https://www.mondadoristore.it/?
• FNAC (FR):?https://www.fnac.com/?
• Bol (NL, BE):?https://www.bol.com/??

As fascinating as the possibility of AI narration is, and as impressive as the quality of cloned voices is (becoming), the real obstacle is distribution.

You have three options:

1. If you want to "record" in your voice using an AI narration tool, Kobo is currently your only option for distribution. This distribution is very narrow and doesn't reach the largest retailers though. I anticipate over time, more retailers will eliminate this barrier as technologies to verify the author's voice and measures to secure sufficient listening quality are further developed.
2. If you are ok using a pre-defined voice, Google Play gives you the widest distribution. You can download the file and upload it on Kobo as well. If the quality is good enough, you might be able to have one of Apple's partners approve the file as well. I doubt that's a sure-fire strategy though.
3. If you want your book on Audible, you need to record it traditionally and distribute via ACX. Once access to KDP's Virtual Voice is out of beta and available to all users, you can get your book on audible, using their pre-defined voices but it will be unedited and using a pre-defined voice, not your voice. I doubt KDP will allow the use of cloned voices any time soon.

?? Be careful with rights and exclusivity clauses preventing you from publishing on multiple platforms!

Will consumers know you used AI?

At this point, publishers must provide information about the use of AI in the book's metadata (ie. narration contributor or AI used for editing or images). It seems all platforms at this point rely on this information being truthfully provided. In the future, the technical review of the published files will likely include a check for the use of AI.

At this point, Amazon consumers see the information in similar forms to this (example is for audiobooks created using Virtual Voice on KDP):

Wow... this article was a heavy one. Even if possibilities seem less impressive than what meets the eye and you might feel somewhat discouraged that so many of the options underdeliver at this point, you should create your audiobook - yesterday!

There is a reason for AI tools mushrooming at this pace and so many enter the race to create the best tools for authors: READERS WANT AUDIOBOOKS! ??

How about you simply choose to break the norms and become a little more personal in the opening credits when you record your book?

Who says you can't deliver a short personal note letting the audience know, you like to experiment and want to give them the audio version as soon as possible and use AI narration in the production of your book?

I bet people are just as curious as you are on this subject and will want to listen to your book even more! - provided you ensured a quality worth listening to they will forgive a few weird pronunciations.

EXTRA TIPS

• Watch out for exclusivity and check if you, as the publisher, keep the rights to publish on other platforms. This is especially important if your narration/distribution platform does not distribute to all retailers.
• Certain parts of your books should not be narrated, ie. the table of contents. You will also want to add files at the beginning (opening credits) and end (closing credits) with information about rights, who narrated etc. Go listen to 4-5 bestselling books and do something similar. Don't overthink it.
• Read your book out loud before even finishing your script. With small adjustments, you'll be able to create your audiobook much faster. For example, a few extra words may be needed to make lists sound natural (ie. an "and" before the last bullet in a list).
• Think about the audio version of your book when deciding on your design concept for the print book. Text boxes and other techniques to highlight specific sections can be troublesome in audiobooks because they break the flow, especially when using AI narration.
• If your book was designed in ie. InDesign, the final version may not be an exact match to the text file you sent to the designer. You may have changed a few words or taken out a paragraph to make the text fit each page nicely. In that case, what you will have received from the designer is a print-ready PDF. Some narration tools can work with an ePub or directly from a PDF but if it can't, use https://smallpdf.com/pdf-to-word#r=convert-to-word to extract the final text to a Word file or ask your designer to export as a Word document.
• When you have exported the audio files, send them to an audio editor to enhance the sound and add the necessary metadata. In my previous book, I could hardly hear the difference in quality (using the above-mentioned Descript method), but there are certain requirements for pause before speech and other stuff you really don't care to learn. Have a specialist to that, it's a small investment.

?? OTHER AI NEWS

International markets have become easier to reach with video content - and you don't need to know the language anymore! Use HeyGen's text-to-video tool and promote your book in Spanish too! If you are not happy with the voice, combine it with https://elevenlabs.io/ to improve it.

The?Federation of European Publishers?in Brussels has issued a statement in which they welcome the adoption of the Artificial Intelligence (AI) Act by the European Parliament. You can read the statement here. AI is becoming more and more regulated, which I think we can agree is a good thing.

The rise of AI means we can replace programming languages with human language prompts thus enabling everyone to be a programmer, says Jensen Huang, CEO of Nvidia.

Singapore will pay for its citizens aged 40 and above to go back to school in light of so much knowledge and jobs becoming outdated as a result of AI.

Want to learn more about AI & Creator tools? Join the Podfest Expo 2024 masterclass (virtual) on April 19.

Or read the book Shimmer, don't Shake: How Publishing Can Embrace AI.

?? The Short Rebel News ??

• As you probably guessed from this article, the book 'How to Become a Nonfiction Author' will soon be available as an audiobook!
• It will also be available in another format, we haven't published in before! Would you be interested in first access for $1? Connect and send me a DM!
• The book writing retreat in May is SOLD OUT. We are currently considering when and where to host the next one and would love your thoughts on that!
• I am hosting a meet-up for self-publishers in Newcastle on June 17, 2024, at 4 PM. I’d love to see you there! It’s the day before Atomicon, hosted by my good friends Andrew & Pete happens. You should check out their event!
• Malene will be walking the Camino Portugues this month! ???? Follow her journey on Instagram and here on LinkedIn.