AI in Game Production: a 2023 XDS Table Topic Discussion

In September 2023, I had the wonderful opportunity to lead a Table Topic discussion at the External Development Summit (XDS) in beautiful Vancouver, a renewed honour after we already hosted one in 2022 . We dug into the exciting topic of “AI for game production success”. If you’ve never heard of XDS, it’s a one-of-a-kind event in the gaming industry that’s been around for over a decade. Its main goal is to bring together major game studios, publishers and external service providers who excel in a variety of areas, such as art, engineering, quality assurance (QA), user experience (UX) and localisation.

I was thrilled to guide this conversation, as we were joined by some of the top experts in the industry. I would like to express my gratitude to our participants:

• Aaron Lemay , Studio Director at Free Range Games
• Curtis Churn , Senior Manager, Audio Direction at Riot Games
• David Rossellat , Partner at CodeLink.io
• Jon Hammon , Senior Producer at Fire Falcom
• Julien Hamaide , Co-Owner at Fire Falcom
• Maxim Fleury , Studio Asset Art Lead at Guerrilla
• Nikki Roth , Senior Technical Artist at Google
• Rebekka Muntola , Art Producer at Metacore
• William Schleifer , CEO and CTO at Powerup

Our lively discussion centred on the fast-paced progress in AI-assisted art and its potential to change the landscape of game development. With this article, I aim to share the many insightful perspectives that have come up during our session.

The manifold use cases of AI in game development

After introductions, we looked at the challenges tied to the use of data and open-source resources and then explored the intriguing world of artificial intelligence (AI) – starting from its adoption to the workings of supervised learning and generative AI, and we even broached the subject of unsupervised learning.

We brought quite a few AI applications into the spotlight, ones that have seen real-world success in our industry: from using AI to create 3D meshes and textures from natural language descriptions (a pioneering project at Endava back in 2021 that brought about substantial improvement in our graphic design department) to more recent implementations using Stable Diffusion, Midjourney, Nerfs or Gaussian 3D spatter.

We also talked about other AI uses, like detecting bugs in code, adapting reinforcement learning agents to changing game environments, pinpointing exploits in games that could give players an unfair edge and implementing generative adversarial networks (GANs) in bots to detect errors in graphic assets.

What’s already going well?

While examining these use cases, we also wanted to hear our participants’ opinions, of course. We discussed what is already working well in the AI ecosystem, areas that need improvement and our individual wish lists for the future of AI in gaming. Jotting down these insights on sticky notes and arranging them on the table, we formed a vibrant collage that reflected our collective hopes, realism and aspirations.

According to the participants of our XDS table topic discussion, the evolution of AI technology has been largely positive, one key highlight being its lightning-speed advancements, as observed by Aaron. This development is unlocking unique opportunities while reshaping the business landscape as we know it. The AI ecosystem has evolved so rapidly that it’s often referred to in terms of ‘commoditization’. Its rapid advancement rate is accompanied by an improved ability to train models leading to new terrains in creative design.

This leads us to the contribution of AI to the realm of art. AI’s role in eliminating repetitive tasks and minimising grunt work in the development pipeline stood out as a significant improvement. Several participants highlighted that AI has considerably boosted the creation and inspiration of 2D art. This also extends to aid for code generation. Furthermore, Curtis noted that AI has enhanced generative voice-over (VO) quality, another area currently gaining attention in the world of game development.

We also agreed that AI is a great learning tool, and there is an increased awareness and vocabulary around it. The growing adoption of open-source models strengthens accessibility, fosters collaboration and opens up ‘blue oceans’ of opportunity, as pointed out by Rebekka. This trend goes hand in hand with AI’s capacity for increased creative exploration, which facilitates the generation of innovative ideas for both seasoned and more junior developers.

Where’s room for improvement?

While the evolution of AI in game production has been incredibly positive, our discussion also highlighted several areas for improvement:

One major theme is around the legal and ethical aspects of AI, particularly regarding legal clarity, licensing, protecting intellectual property (IP), attribution and awareness of responsibilities. Julien noted that the black-box nature of AI and its possible implications were other areas of concern. Potential monopolies in AI development have also been identified as obstacles.

For example, there are challenges around data and training that need to be addressed. Attendees noted that the process of training models and securing training data is currently resource-intensive and possibly also comes with legal or moral problems, suggesting the need for more efficient training methodologies and better sourcing of data.

Our participants expressed wishes for a solid legal framework that ensures the fair treatment of all parties involved. This would encompass legal agreements that benefit both talent and studios, for example in voice-over generation, and allow contributors to be compensated appropriately for the work that goes into the datasets.

Another point highlighted was control – or the lack thereof – when it comes to AI-generated output. Specifically mentioned were areas like having the flexibility to guide the model of VO direction and track VO generated for accounting purposes. The panel also noted a lack of specificity when using AI tools and concerns regarding the ethical training of datasets.

AI’s role in revolutionising IP rights was also discussed. The dream is to future-proof existing IPs by training AI with internal assets or synthetic data. At the same time, concerns were voiced around the need for robust regulation to secure the use of AI in outsourcing scenarios. In William’s vision, blockchain technology should be used for attribution and reliable remuneration, indicating a belief in the potential of combining advanced technologies for enhanced transparency and fairness.

The discussion around securing IP evolved into a broader issue: the lack of clear regulation. Everyone agreed on the importance of tracking AI’s evolution and ensuring proper regulation . This way, the industry can provide adequate security safeguards while actively combating biases. Nikki pointed out that it’s essential to create an unbiased AI ecosystem that caters to everyone equally, respecting individuality and diversity. Addressing issues related to stigma and bias also emerged as crucial to ensuring that AI benefits everyone. This involves not just deploying AI responsibly, but also making sure the technology itself is unbiased, fair and just.

Attribution was another legal aspect brought up for discussion. Who gets the credit for AI-generated work if the original source can’t be traced back? And how can artificial intelligence be held accountable for any unintentional wrongdoings? This forms a significant part of the ongoing debate around the legal and ethical implications of AI usage.

?Accessibility was another topic broached by our participants. With AI development leaning towards a lot of complexities, ensuring that we are able to democratise AI technologies is recognised as an important challenge to tackle.

Finally, attendees expressed the need for more diversified models in AI development. The more variety we have, the more comprehensive and inclusive the AI solutions can be.

A glimpse into the future

Our AI roundtable surfaced exciting ideas about how AI could shape the gaming industry in the next years.

Participants conjured up holographic solutions, reminiscent of Cortana from the Halo series. This shows a shared excitement for integrating AI more broadly and igniting a renaissance of fun and creativity in gaming. Alongside this, they hope for AI that can transform conventional methods and find new ways to approach challenges, signifying great expectations for AI to enable change across the industry.

The participants also voiced their wish for “magic-fast training” and better integration with existing tools, hinting at the potential for improvements in training efficiency and tool compatibility. They keenly stressed the need for full customisation, reflecting the demand for AI tools that are adaptable and sensitive to the unique needs of their users.

Even though AI’s rapid evolution can sometimes be hard to keep up with and confusing, it was acknowledged that the possibilities it opens up in the world of game art make the learning curve worth it.

The participants also contemplated how the advent of AI could impact job markets. They shared the hope that the AI revolution would catapult human creativity to unprecedented heights, rather than diminish it. This was reflected in their desire for personalised games, a prospect made more feasible with the advancements in AI.

One captivating notion that beautifully illustrates the group’s shared hopes is an AI that could create art by reading human minds – an idea that showcases the attendees’ desire for AI to seamlessly integrate into human thought and creativity. This emphasises the dream of AI as a tool which – rather than replace human talent – unlocks doors to previously unimaginable experiences and efficiencies. In essence, a tool that ensures more fun for all, as one participant put it.

An exciting journey ahead

In our lively XDS Table Topic discussion, participants recognised the giant leaps AI has taken to transform the gaming industry. Yet, they didn’t shy away from acknowledging the bumps along the way, like the need for clearer legal guidelines and enhancing AI control. The progress we’ve seen in areas like tool integration, customisation, enhanced training and inventive applications of AI painted an exhilarating picture of the future.

The participants let their imaginations fly, dreaming of things like holographic tools and even creating art with pure thought. These responses are a powerful testament to the potential of AI and that it could launch our creativity into realms we’ve never encountered before.

When it comes down to it, the genuine engagement, open attitudes and constructive exchanges promise that we’ll be able to navigate this thrilling AI revolution in gaming while keeping our ethics intact and our creativity thriving. Together, we can steer this ship effectively, ethically and with a bold dash of creativity.

Appendix: the poll results in detail

?The results of our XDS table voting pool were as follows:

1. AGI will be a reality (next 20 years): With 81.25% agreeing, this aligns with the optimism in the discussion about AI’s rapid advancements. The progress in AI-assisted art and its transformative potential in game development, as highlighted by experts like Aaron Lemay and Curtis Churn, supports this belief. The remaining scepticism and uncertainty, represented by 6.25% “No” and 12.5% “Undecided” votes, stem from the ethical and legal challenges, such as AI’s black-box nature and potential monopolies.

2. Reliable 3D AI solutions (next 3 years): 75% agreement reflects the confidence in AI’s near-term impact, resonating with already successful AI applications like 3D mesh generation from natural language. Concerns (12.5% “No”) relate to the control issues and lack of specificity in AI tools, while the undecided votes (12.5%) reflect the concerns over the rapid commoditization of AI and its implications.

3. AI will completely render a game (next 5 years): With 37.5% agreement, this statement might mirror the cautious optimism about AI’s role in game art and voice-over quality enhancement. The higher disagreement (43.75%) and undecided responses (18.75%) reflect the concerns over control, ethical training of datasets and the legal aspects impacting AI’s reliability to fully render a game.

4. Anyone can art direct (next 5 years): The poll reflects a split opinion (31.25% “Yes”, 50% “No”), which is due to the diverse viewpoints on AI’s role in creative tasks. While its capability to aid code generation and 2D art creation is recognised, the lack of control and specificity in AI-generated output might fuel scepticism.

5. AI that can generate production-ready 3D characters (next 5 years): The 68.75% agreement aligns with the enthusiasm for AI’s role in creative exploration and innovation, especially in art and character design. The rejection (12.5%) and uncertainty (18.75%) were attributed to the challenges in training models and securing training data, alongside the ethical concerns in AI use.

6. AAA production-ready characters using NLI (next 3 years): The balanced responses (43.75% “Yes”, 37.5% “No”) reflect the current view where AI is seen as a great learning tool with increasing adoption, yet it’s facing challenges in data training and the need for more diversified models, as indicated by participants like Nikki Roth and Rebekka Muntola.