I used ChatGPT and Stable Diffusion to make an entire AI video game. Here's how it went.

You can see the whole process covered in my newest YouTube video, above, or read on.

I'm not an experienced programmer, so I picked a genre that I thought played to generative AI's strengths: a point-and-click adventure game! Lots of storytelling and images.

Story and Visuals

That proved to be the right choice. I asked the fiction writing A.I. Sudowrite to give me possible settings, and it immediately came up not just with a creepy sunken submarine, but also with a title (Submerged Secrets: A Haunted Submarine Adventure), story, and all the rooms aboard and items I could find there.

I then converted these room descriptions into Stable Diffusion prompts - with my own brain, not AI, I'm afraid - and had my whole world visualized in ca. two hours.

I edited some of the more difficult scenes like the Bridge (which needed a removable giant squid) with the techniques I developed in the past few months on my YouTube channel.

Coding

ChatGPT then helped me set up my basic scene in the open source Godot Engine. Not only can it code a little GDscript (similar to Python), it also understands the basics of "node trees" - the fundamental structure of a Godot game.

It also came up with some basic actions it wanted to see in the game.

The ball was rolling! I had my story, I had my basic interface...and then I had to put on the brakes. The GDscript ChatGPT was putting out was...problematic in several ways. Among other things, it started calling libraries and functions that just plain don't exist.

The development slowed to a crawl. I hadn't set up my structure very intelligently, and didn't have enough dev skills to understand how to fix it. Before, I thought I'd already done the hard part. But suddenly I spent hours just debugging and googling my way along...

ChatGPT was no help here. I was only using it to code basic helper functions that converted some variables back and forth, nothing more advanced.

I'm sure a lot of it was due to my choice of language and it could've coded more in actual Python, for example, rather than GDscript. But if ChatGPT wants to be more useful, it has to start supporting the structuring of code across multiple objects, not just short little bits of it.

Results

AI for now remains *incredible* as inspiration for both storytelling and visuals. Knowing the limits that I know now, I would definitely use it again in the future - just with lower expectations.

If you want to see my process in more detail, check out my video. The resulting game is occasionally a bit buggy, but fully playable here:

If you thought this was interesting, share this article and subscribe to my channel to help me do more experiments like it!

Here are all of the tools I used: