The State of Generative AI in the Video Gaming Industry

Special guest co-author William Grosso , CEO of Game Data Pros .

Artificial Intelligence (AI) has been making significant strides across various industries, and the video gaming industry is no exception. According to a survey by the Game Developers Conference (GDC) involving over 3,000 game developers, one-third of them are incorporating AI into their development processes (NPR article ). This integration of AI spans multiple facets of game development, ranging from content creation to enhancing user experience. If you’re looking for directional guide on what’s possible, the Andreessen Horowitz article is a great primer. ?However, for the state of what’s actually being done in the gaming industry today, H1 2024, here's a closer look who is actually using generative AI and how.

?Overview

The digital entertainment industry faces two significant barriers in adopting AI within a naive SaaS model, primarily related to quality and intellectual property (IP). The specific challenges include:

- Realism issues in generated images and videos, such as anomalies like people with six fingers

- Content generation errors, including non-existent gameplay and loss of user history context

- IP leakage and loss of control and ownership

- Confidential information leakage

Recent controversies involving Slack, Zoom, and Adobe Creative Suite retaining client data to train their AI models underscore the validity of these concerns.

However, high-end game studios are not entirely shunning AI. Instead, they focus on "constrained" AI use cases to enhance efficiency while mitigating these risks. For those less concerned with quality or IP, the faster time-to-market enabled by AI becomes increasingly attractive. In contrast, smaller studios and games with fewer IP concerns are more willing to explore the boundaries of AI use cases.

1. Content Pipeline

While AAA games, which can cost hundreds of millions of dollars to produce, are hesitant to adopt generative AI due to the associated risks of unknown and unpredictable technology, AA games and smaller studios have started integrating AI into their content pipelines.

These studios leverage generative AI to expedite content generation in specific areas that are bottlenecks. ?One notable example of a content generation tool is ZooBuilder, a 3D animator that animates four-legged animals based on videos. This tool helps designers accelerate the most tedious parts of their process, allowing them to focus on more creative and interesting tasks (CNN ).

Another example, AI-generated voices for foreign languages enable games to be released internationally without the previous costs and pipeline delays. In addition to pure efficiency, Hasbro has launched an infinite version of Trivial Pursuit leveraging AI capabilities directly in its game play (Forbes ).

2. Brainstorming and Prototyping

Generative AI is also being utilized in the brainstorming and storyboarding phases of game development to speed up the creative process without introducing IP related risk. It helps in generating ideas, designs, and iterations on different paths to quickly narrow down to the final creative approach. ?When it comes to final creatives visuals and content, the full production is still fully owned by human creatives. Similarly, prototyping is another area where AI speeds up the creative development process. The lack of IP protection and clear data lineage introduces too much risk for game studios to use any AI-generated content in the final game. Even tools like GitHub Copilot, which generate code, are only used for service-side code and not for any end-user device code to minimize potential IP claims risks.

3. Minor Modifications to Gameplay

While key gameplay elements, characters, and dialogue remain under the purview of human creatives, AI is increasingly being used to generate dialogue for Non-Player Characters (NPCs) to enhance their personality but only when the dialogue no impact on game play like issuing quests. Platforms like Inworld and techniques developed by companies like Square Enix, Replica Studio, and Ubisoft ensure that NPC dialogue can be generated and moderated effectively. For example, Ubisoft's La Forge has developed Ghostwriter, a tool that helps generate NPC dialogue. They also created a technology in 2022 that generates NPC gestures based on speech input, ensuring natural gestures that match the dialogue (Forbes , CNN ).

Additionally, advancements in hardware have enabled real-time voice interactions with NPCs. Nvidia and Convai demonstrated a real-time demo showcasing this capability at the recent GDC conference (NPR ).?

4. User-Generated Content (UGC)

Even though many games allow for user-generated customizations of their characters, true UGC platforms are ones where the game play has support for extensive game play customization like Minecraft, Roblox, and Fortnite. Andreessen Horowitz ? predicted that generative AI will affect UGC platforms in 2 phases, phase 1 for tooling improvement, and phase 2 for complete reimaging of the creation process or even the games that gets created. The state of generative AI in UGC game platforms right now are still squarely in the tools phase. The Roblox Assistant as an example leverages a chat interface using LLM’s to help users create custom avatars faster and easier (Roblox ). Generative AI platforms like Scenario.gg exists to help create custom asset generation specific to games and art styles, but it’s still only available to game and content creators, not yet to the players. In related segments, Interactive fan fiction platforms like Hidden Door are also exploring generative AI to enhance user creativity (GDC AI session). What the Andreessen article doesn’t speculate on however, is whether AI will essentially kill UGC as a revenue stream or a differentiator when we move to phase 2. If AI makes UGC creation drop-dead simple, why would anyone pay?

5. Open-World Game Content

Open-world games are beginning to leverage generative AI to create content and levels dynamically as players explore. Although traditionally this has been achieved through procedural generation techniques in games like “No Man’s Sky”, research at Rikkyo University in Tokyo is ongoing to develop AI that can generate complex environments such as mazes and dungeons in real time? (CNN ). An example of this was Modl.ai ’s work on level generation with large language models that was presented at the recent GDC AI Summit. There are rumors that Microsoft is experimenting with allowing full builds in Minecraft to be controllable through a chat-only interface leveraging LLM’s, but not announcements yet on timing for general availability (Forbes ).

More speculatively, it has been conjectured that the true “VR Revolution” is going to be enabled by AI-generated virtual realities. Creating a 3D immersive world the old-fashioned way (manually) is simply too much work. But the next generation of AI tools is likely to make it much easier. Imagine the marriage of Apple’s Vision Pro with next-generation AI-based multiverse generation, it just might be the thing that causes all of us to sit on our couches wearing goggles.

6. Content Moderation and Toxicity Detection

AI isn't just enhancing the creative aspects of gaming; it's also being used to improve the social environment within games. With LLM’s capabilities around understanding text, images, audio, it’s being used to help with content moderation of user content on a much larger scale. Recent session at GDC of AWS Bedrock in partnership with Sony demonstrates the ease of which this can be added to the current content moderation process to improve both the speed, and the quality. In addition to behind-the-scenes optimization of the content moderation process, NPCs can be equipped with AI to detect and moderate toxic behavior among players. Imagine an NPC who in behavior acts like another player interacting to discover bad actors in a multi-player environment. This dual functionality of AI helps maintain a positive gaming experience while ensuring safety and inclusivity.?

Conclusion

The potential of impact of Generative AI on the gaming industry is predicted to be revolutionary because of the content generation capability for every modality that’s evolving at break-neck speed. However, the adoption is slowed because of the unaddressed IP, and legal risk as well as the lack of integration into the existing toolsets and processes. Generative AI is starting to hit maturity with text generation capability, and still growing into visual, audio, 3D generation capabilities and the current usage is still squarely in the phase 1 adoption stage that’s focused more on the tooling, spot efficiencies, and peripheral game assets.

In terms of technology adoption, game developers prefer using open-source Large Language Models (LLMs) with open training data disclosed for generative AI, as that provides some assurance on data provenance. Amazon Bedrock in conjunction with open-source LLMs are becoming the preferred choices for vendors due to their transparency and reliability.

The integration of AI in the gaming industry is multifaceted and continues to evolve. Smaller players take more risk and may become the testing ground for new processes, tools, and platforms as the technology matures at the same time. The timeline is uncertain, but what is certain is that the full gaming landscape of game studios can completely change as a result of Generative AI.

If you like more content like this, subscribe to my substack: yuying.substack.com