List & Analysis of AI Tools in the Creative Process in 2024
A Research Dive into AI Software Organized by Different Stages of the Production Workflow
What is this about?
This article shares a list of AI software categorized by different stages in the film, TV and games pipeline, as well as some of my initial thoughts on AI. I've dabbled, but also at times hesitated, because, like many of you, I care deeply about creativity and the artists who bring it to life. While AI can offer benefits for small teams, there are still important questions about how it’s trained and its fairness to creatives.
Now, I’m writing this because I love learning & sharing, especially about pipelines, and many people have asked me about AI tools. While I’m still figuring things out myself, I believe it’s important to share so we can all explore it together, even though it’s a complex topic.
AI is already influencing the creative industry, and it’s becoming something we might need to understand artist. Over the next few years, knowing how to use these tools will be necessary , but that doesn’t mean we have to lose our creative spirit.
This article isn’t about promoting AI, but rather sharing a list of software and offering some reflections on its potential impact on the creative pipeline—both the risks and the benefits. My primary interest lies in whether AI, when used thoughtfully, can help small teams push their ideas further before seeking outside investment, potentially preserving more creative control and independence in the process.
AI Workflow, Communication & Management
Software that helps connect different parts of the pipeline, connectivity amongst teams and manage assets more efficiently. I’m particularly interested in tools that are node-based, visual ways to work with these complex systems.
One standout is Comfy, an open-source AI workflow developer. Other workflow creators caught my attention, but nothing stood out quite as much in terms of unique functionality and a unique mission.
AI Image & Video Generation
AI-generated images and video from text prompts, which, as a visual artist, feels both exciting and intimidating. The way these tools are trained raises concerns, but they also present overwhelming possibilities for artists and small teams.
I think it's important that we all start learning these tools, as becoming proficient in them could be necessary to stay competitive. I still believe artists hold the creative power, but studios may begin to expect AI in the early stages like pitch decks and concept creation.
The stand-outs here are Sora, Runway, and Dream Machine, though there’s still a lot to explore. Leonardo and Adobe also make the list, so there’s definitely more homework ahead.
AI Interactive Video
Software where different imagery can be fed into a system to create evolving videos, or even allow live web-browser interaction with a video. As a tool to iterate quickly on early ideas, the potential to feed concept art created by you and see it evolve is especially intriguing to me.
The ability to create evolving, interactive moving stills in a web browser feels like something we can't do with other tech. It also seems like a promising avenue for early-stage development and iteration.
Pikachu, Batman, is that you?
I wanted to highlight Determin.ai from the Interactive Video section because it stood out to me in an unexpected way. As a huge Pokémon and Batman fan, seeing Pikachu used here felt a bit unsettling, even though when artists repurpose well-known characters I’m cool with it.
The ability to input anything into this tool feels both powerful and exciting, but seeing some of my favorite brands being used like this is a reminder of the broader concerns around AI-generated content. It highlights some of the challenges we face when dealing with AI, where the lines between homage, repurposing, and originality can blur.
AI Content, Compositing & Editorial
Tools designed to speed up the compositing or editorial process, with Wonder Dynamics standing out as a dominant player. While it’s seen as a potential game-changer that could disrupt the VFX industry, it still has room to grow to match the expertise and artistry of our incredibly talented VFX professionals.
This shift could understandably lead to concerns about job security, but it also opens up new opportunities. As automation takes over more repetitive tasks, it could allow artists to focus on higher-level creative problem-solving and storytelling, bringing even more meaning and innovation to the work we do.
AI Virtual Worlds & Game Worlds
Virtual backgrounds, including matte paintings and virtual worlds for video games and virtual production. Some interesting developments are emerging that could soon impact how we create both real-time and VFX environments.
After reviewing these tools, it's clear that while AI world-building is advancing, there’s still a lot of refinement needed to achieve the complexity and detail required for the worlds I’ve led in virtual production projects. Places like Cuebric are tackling the challenge of 3D world creation, but much of the process remains hands-on and will continue to evolve.
AI 3D Asset Creation
3D asset creation, is an area that was a huge eye-opener for me. I hadn’t explored enough in this space before, but after seeing the potential for quick early-stage exploration, and photogrammetry cleanup, it’s clear there are significant time savings here, even though achieving high-quality results remains a creatively driven, hands-on process.
After looking into these tools, I see how they could drastically impact VFX and the broader industry. How this impacts larger and small productions is yet to be seen, and I think it's going to be one of those tools that will be expected in production.
领英推荐
AI 3D Animation, Rigging & Mocap
I am intrigued by Motion capture (mocap) and animation tools, features like secondary physics, and features that help pose characters more organically, seem pretty neat.
After looking into these, it's clear this is a fast-evolving area, with key players already using AI to drive animations procedurally or through commands. It’s something worth testing and keeping an eye on.
NPC Agents
NPC agents, where AI-driven characters respond procedurally to interactions. I found it particularly interesting how these agents integrate into world-building, creating an immersive experience where AI characters essentially act as people you can engage with in pretty much any kind of setting.
After exploring these tools, I can see that while they’re making progress, they still fall into the uncanny valley, making the experience somewhat unsettling. However, I believe this technology will evolve quickly, and it’s an area uniquely suited to AI, as no other approach replicates this kind of interaction.
Hair | Cloth | FX
This section focuses on hair, cloth, and effects, which are traditionally some of the hardest elements to handle in CGI. I didn’t find much in this area, but I believe AI could help significantly with complex simulations like water or intricate effects that we currently handle with tools like Houdini.
I’ve seen examples of people using AI to create FX, but ZIBRA stands out as an option that seems to have a focus on visual simulations. However, it seems like there’s still room for further development in this space. I welcome any suggestions in software I can include in this section.
Speech | Audio
AI-generated music and audio tools may provoke mixed reactions, especially since music creation feels so personal. While these tools could be incredibly helpful for a visual artist like myself, there’s still a certain romance and emotional depth that only a human touch can bring, which I feel is still missing.
These are tools I will be looking into for early development around an idea, which may inspire new creations. However, I feel it’s already impacting the music industry in ways I don’t fully understand, as it's not my native field.
Game Development
This section focuses on AI tools for game development, ranging from machine learning to more recent innovations where you can input text commands, and elements appear directly in the game—reminiscent of Scribblenauts. While this section is interesting, I haven’t yet seen an AI platform that can fully create a 3D game from start to finish in an all-in-one solution.
A comprehensive high-end AI-driven game creator still seems to be missing. There’s potential in this space, but as of now, it doesn’t seem to fully meet the demands of game development.?
Wait what? How about the humans?
Well, this one stood out. This one feels like it says the quiet part out loud—videogame.ai is a good example of why artists are concerned with AI technology. But, don't be alarmed, it's not that easy, and I do look forward to seeing where these studios take this, as right now everything seems experimental.
Studios
Studios using AI to produce films or offer specific services. While AI presents some intriguing possibilities, there’s something unique about the handcrafted essence humans bring to filmmaking that’s hard to replicate.
I’ve spent some time watching many of these AI-generated films, and while they serve as a very early starting point or reference, they lack the depth and emotion found in human-created work. Even as a reference, I find using traditional films far more impactful.
However, I believe the real potential lies in blending hand-crafted skills with these tools, and I expect we’ll see some truly high-end films as artists begin to master that balance.
?
Website Creation
Website creation tools, as all the work we do ultimately need a platform to showcase it. I explored several options, and while it's interesting to see how accessible website creation has become, it still feels quite limited at this stage.
There are many companies in this space, including major players like Wix and Squarespace, though I didn’t include them in the list. Overall, while these tools offer a great starting point, AI-hosted sites still feel quite basic and lack the level of customization needed to create a truly unique page—that’s where a talented designer shines, hats off to them.
Final Thoughts
After exploring this software, I see the tools are here, and artists can build around them—but I understand the concerns. I’m experimenting with AI to see if it can help us maintain more creative control, but I also understand the anxiety around job security—AI is already potentially replacing roles, and we can’t ignore that.
While it’s scary at times, there’s also potential for us to reimagine how we work and stay competitive as the industry changes. I hope sharing my learning helps us navigate this together. Thank you for reading.
Investor | VC | Advisor | TEDx-Speaker | Enabler
6 个月AI stimulates imagination, yet empathy guides true creativity.
WordPress Developer | Shopify Developer | Freelancer | Wix Developer | Desktop Support Engineer | IT Executive | Learning DevOps | AWS & CI/CD Learner
6 个月Interested