Creating a Gaming-AI with Reinforcement Learning
Andreas Kretz
I teach Data Engineering & match talents with dream jobs | 10+ years of experience | 3x LinkedIn Top Voice | 200k YouTube subscribers
I’m a passionate gamer ever since I was a kid. I started with Super Nintendo (SNES), then I went into computer gaming. For some years now, I’ve been playing on the PlayStation.
So, the gaming industry is very near to my heart.
That’s why the appliance of data science in the gaming industry, especially AI for bots, or AI for computer games in general, really interests me.
Over the past week I’ve been looking into how to get started.
Creating my own AI I can play against.
OpenAI & Elon Musk
It turns out this is actually a big trend. The company OpenAI, co-founded by Elon Musk, is especially big into that.
There’s OpenAI Gym, like gymnasium and OpenAI Universe. Two tools where you can train your AI. On for instance Atari games, or flash games, or browser games, or with Universe any game you want.
It’s very interesting. I’ve been looking to find a niche that interests me a lot and that I can deep dive into. I think this is it.
I always tell people, that they need to find a project that excites them, and just start. Projects help you get practical experience.
So I’m, going to look at selecting a game and then trying to build a AI for that game. The long-term goal for me, is to build in an AI that I can play against.
Reinforcement Learning
How reinforcement learning works is, the algorithm plays one round of the game. After that round he gets a feedback how he performed.
If the performance was good he gets rewarded. If he performed bad, he gets penalized. This then influences the building of the next model.
This feedback lets the AI tune itself over time, so it will hopefully learn and perform better.
Schooled or Apprenticeship Learning
One way to train the algorithm is called schooled or apprenticeship learning. You are teaching the AI by playing the game and letting the AI watch.
The AI then learns and after the schooling phase you let the AI play and learn by itself. The upside of this is, that the algorithm already has some idea how stuff works. It doesn’t start from zero.
Learning Completely From Scratch
The other way is that you start completely fresh. You don’t teach the AI, you don’t let it watch how you play.
You just let it play and let it fail, and fail, and fail. Over time it will learn. Hopefully.
How OpenAI Universe Works
Universe lets you play and learn any game.
How that works is you have two machines: One machine where the actual game and a VNC server is running on and another one where you learn.
The machine where the game runs also has a VNC server. This way the game can be remotely controlled from another computer. What that system also does is, it delivers the performance feedback for the reinforcement learning.
The second system is connected to the VNC server and plays the game. It is also running the actual algorithms that train the AI models. It gets the feedback for the reinforcement learning.
Deploy In The Cloud
Both machines are dockerized, so you can deploy them in the cloud if you want. So you can run the whole process either your PC at home, or you can use cloud instances.
If you like me don’t have a huge gaming system with strong GPUs at home this is the best choice. I for instance only have a MacBook Pro and that is not optimal for for building machine learning models.
Just get two cloud servers and run the game and the actual building of models, the actual learning there. You can deploy for instance on GPU instances to run and train the algorithm. This way you don’t need to use your own computer.
It makes it very easy and cost efficient for beginners, because you don’t need to buy expensive hardware, like Titan X’es or so. You just rent the computing power in the cloud, use it and when you’re finished, switch it off.
Watch the AI Play & Give It a Hand
While, the algorithm plays the game and learns in the cloud, you can also watch how it performs. Over the VNC connection you can connect to the instance where the game is running on.
Watching the algorithm play and checking out how he learns very interesting. you can also influence what is happening by taking over the controls and giving the Ai a hand.
This makes sense if something isn’t working right, or in the case of apprenticeship learning.
Learning Data Science By Doing
The cool thing is that you have an environment that you can easily set up, work and learn with.
You will learn to code in Python, how to use for instance tensorflow, get hands on experience with deep learnng and so on. All that by having fun while createing an AI that is hopefully able to learn and play a wide range of games.
You can start with pong one of the all-time greatest and it goes up until Doom, or I have seen on YouTube guys that create AIs playing Grand Theft Auto 5.
It’s a wide range of games, very, very interesting sector. I read on Wikipedia OpenAI has been collecting about one billion in funding.
It’s a crazy effort to make that technology public, to give that chance and power to people.
.
Partner Alliance Marketing Operations at Data Dynamics
11 个月Your passion for gaming and data science shines through in this insightful post. Building AI for gaming using reinforcement learning sounds like an exciting venture, especially with platforms like OpenAI Gym and Universe opening up new possibilities.
Management consultant / process optimization specialist
7 年How did you determine your algorithm is male?
Helping companies on their automation and AI journey
7 年Very interesting stuff! Slightly different angle but a friend of mine who is a games developer has told me this is really the art of addictive gameplay. Getting the level of the challenge right for the user. Even for the fairly simple game he built if you really knock it out of the park on one level the game becomes more challenging for the next and vice versa. It doesn't really have AI with reinforcement learning or anything fancy, I imagine it would be something simple, kind of like this (this is how it works in my head anyway): - if time_to_finish_level < 3 mins => difficulty_adjustment + x; - if time_to_finish_level > 6 mins => difficulty_adjustment - x; What you are going to do sounds awesome!
Healthcare Senior Data Analytics Analyst | Statistician | Data Engineer | Analytics Developer | Data Scientist | ROI Analysis | Clinical Analysis | RAF Analytics | Visualization
7 年I am a avid gamer, so I see bots all the time in my MMORPG game, now to think of it, they are AI bots. The bot system is a big money making business , which is not fun to deal with when playing for leisure. All story aside, I like your article , thank you, gives an insight into generating bots. It is already in advanced implemented in the gaming world. So, making into a project is good idea but with an idea of improving gaming exp would make a difference. Just a thought. Thanks.
Andreas I would very much appreciate being able to learn from your experience by following the progress of your project and being able to review your implementation, so please do make it a project and share it publicly. Thanks!