Screaming Frog and Scaling AI Supported SEO
The one issue I've had with all the AI discussions is just how tedious many AI workflows are. AI tools are meaningful to me if they do 1 of 2 things: save me time on tasks I am already doing or enable me to do things I wasn't able to before. A lot of AI workflows I've seen are just shifting time from one aspect of work to a different aspect of work and I don't see that as a net gain.
Take for example having ChatGPT write meta descriptions for your website. Say you have 800 product pages that need meta descriptions. The old way of doing this is creating a basic template of the description and using some Excel formula magic to concatenate the product name and variation into the meta description.
The ChatGPT way is pasting the product description into the tool and asking to have a meta description written, then copying that out of the tool into your spreadsheet. The description might be slightly better than the Frankenstein version but the time it takes to do this process for 800 products is time-consuming and almost comparable to just doing it manually.
The same applies to page titles, open graph descriptions, image alt tags, and dozens of other elements for the webpage.
Screaming Frog, a well-known web crawling tool that allows you to visit every page on your website and collect data from it has had a revolutionary update recently. Version 20 now allows you to trigger custom Javascript on every page it crawls.
Okay, let's pause for a moment because if you are anything like me, your eyes may have just glossed over a bit. Javascript is great. I've seen SEO professionals do amazing things with javascript, rewriting titles and meta descriptions on the fly based on information coming from other aspects of the "data layer". I'm not one of those professionals. My Javascript days never got past "Hello world".
Thankfully, you don't need to know anything about Javascript to take advantage of this advancement. ScreamingFrog has added in the code you need to access the OpenAI API and you don't need to do anything more than activate it (and give it the API key... and do some prompting, okay so there are a few steps to this process).
领英推荐
I won't pretend to write a tutorial on this since ScreamingFrog already has:
About 7 years ago I took a survey of the SEO staff I was working with to see how long it took them to write meta descriptions for pages. It averaged 3-5 minutes per page. That moved to the higher end for more complicated service pages, the lower end for product pages that could follow a template. Rewriting an existing meta description was easier than starting from scratch.
I haven't used the ScreamingFrog/OpenAI workflow enough to know how much time it is saving, but I do know that it is at very least for some of the pages giving me a headstart so I'm not starting from scratch. The time savings may be minor over the course of a full engagement with a client, but this process is saving my mental energy for higher and better uses.
In an earlier edition of this newsletter, I pointed out how you can use ChatGPT to evaluate your content based on a persona. Using ScreamingFrog, you can use that same methodology to get feedback on every page on your site. (Don't. You can, but not every page needs that level of scrutiny, just focus on the pages that are clearly in your conversion funnel. You can drive yourself mad trying to make every page resonate with your potential buyers.)
Are you planning on using AI tools to scale your SEO?