Movies and AI

Welcome to my LinkedIN newsletter! In each issue of The Digital Agenda, I share my thoughts on what's hot in applied emerging technologies from the CXO (CDO, CAO, CTO, CSO, CPO) lens covering data, analytics, AI and more innovations in applying emerging technologies that matter for the enterprise. This edition focusses on AI in moviemaking triggered by a recent visit to a popular movie set. Please do share your comments and subscribe if not already done so.

****************

Curiosity got me into this Hobbit hole

Last week I returned from a vacation down under where amongst other outings, I had an opportunity to visit the Hobbiton Movie Set in NewZealand where large parts of the multi-Oscar winning 2001 film Lord of The Rings (LOTR) and its sequels were shot.?

The fascinating stories of Hobbits,? the Shire, the characters and director’s attention to detail and intensive labor that went into the movie making process as told during the nearly 3 hour tour by our guide were simply wonderful.

This also made me think how relevant those practices will be in today’s AI infused world where each day we hear of surreal advancements in visual effects (VFX) and dramatic democratization of movie making via advances in? large language model (LLM) driven automatic video making be it from Open AI Sora or Pica labs or last week’s?GoogleNext announcement on Goggle Vids

Will AI drive down the costs of movie making? Will there be a need for actors when AI agents with emotions can be created effortlessly and anyone's voice can be cloned and even auto translated in multiple languages in real time.?

With those questions, I began to think about how LOTR would have been possible with the technologies of 1999. The surprise came when I dug into their behind the scenes tech. They had used a custom software called Massive for visual effects and crowd character replication autonomously almost 25 years ago! Subsequently the company Massive Software was created to bring this technology to film and television productions around the world. Here is a list of just some of the feature films in which Massive was used.

I was intrigued and upon further digging, found that Massive’s Founder and CEO Stephen Regelous received an Academy Award in 2004 for Scientific and Engineering Achievement for the design and development of Massive, an industry-first system utilizing AI-driven autonomous agents to generate animation.? Stephen had conceived of the basic concepts for the software in the early 90s after studying Artificial Life agent based systems, and developed Massive years later for The Lord of the Rings trilogy, where Massive transformed audience expectations for epic crowd and battle scenes in feature film and television.? Incidentally Stephen began his software career writing solutions for particle animation, muscle based facial animation, morphing and an L-system used to synthesize plant life which has been widely used in the motion picture industry.?

Movie Making Vertical and AI?

The 1968 film 2001 -A space Odesssy introduced HAL (Heuristically Programmed Algorithmic computer), a sentient computer that may have inspired today's voice assistants like Alexa and Siri. Stories of AI overtaking our world and dominating humans like in Terminator have fueled fear of AI in our society for a few decades now. While much of the AI hype recently? has been centered around Generative AI & horizontal capabilities of foundational models, that’s not the full picture. The real opportunity of AI lies not in creating mediocre marketing copy, but in the ability to reconfigure value creation across vertical value chains.?

So today, let's dive into the Movie Making vertical where we will review where all AI has already penetrated and what are the future directions. It's about how AI is revolutionizing the way we tell stories, the way we engage audiences, and even the way we produce and distribute films.

Pre-production

AI's infiltration into the movie industry begins before a single frame is shot.?

• Scriptwriting & Script Analysis: Using deep learning, AI can analyze hundreds of scripts and predict their potential success or failure. This helps screenwriters fine-tune their scripts and optimizes the decision-making process for producers. Tools like ScriptBook use natural language processing to analyze scripts, predict box office success, and offer insights into plot and character development. For instance, 20th Century Fox employed AI to analyze the script of Logan, which helped in making informed decisions about the movie's plot and themes.

• Casting and Location Scouting: AI can streamline the casting process by analyzing historic data, spotting trends, and predicting the next big superstar. Also, it can aid location scouting by providing insights about weather predictions, crowd control, and logistics, ensuring seamless shooting. Warner Bros. partnered with Cinelytic to use AI for casting decisions, evaluating an actor’s market value to predict a film's financial success. For example, let’s look at location scouting. AI algorithms can sift through thousands of hours of footage to identify suitable filming locations, streamlining what was once a time-consuming process.

• Storyboarding and Previsualization: In the storyboarding stage, AI comes in handy in pre visualizing scenes. Traditionally, storyboarding is a time-consuming process, often taking weeks or months. AI, however, can help transform this into a quick, iterative process, allowing for rapid prototyping. AI-enabled software can also create lifelike animations for scene previsualization, helping directors, producers, and production designers visualize scenes before they are shot. These algorithms can design 3D environments and model the actors’ movements within these environments.

• The pre-production phase, once considered the most labor-intensive part of a project, is now becoming increasingly streamlined thanks to AI. This technology not only enhances efficiency but also facilitates creativity, enabling filmmakers to draw out better storylines, create compelling characters, and design innovative scenes.

During Production

AI really is changing the game in film and video production. Let’s break down some ways AI is playing a critical role during the production process

• Automated Camera Systems: One noteworthy advancement in production is the rise of AI-driven camera systems. These automated systems have the ability to capture high quality footage in any environmental condition. Rain or shine, day or night, these machines are up to the task, providing reliability far beyond what traditional operators can. The main advantages to using an AI-controlled camera system include:

• Planned Visual Effects: During filmmaking, AI plays a crucial role in visual effects (VFX).Ever Since being used in Lord of the Rings,? Massive Software has become the leading software for crowd related visual effects and autonomous character animation and appeared in hundreds of productions including Avengers, James Bond movies? and Hunger Games TV series to name a few. Disney’s FaceDirector software can generate composite expressions from multiple takes, enabling directors to adjust an actor's performance in post-production. This technology was notably used in Avengers: Infinity War to perfect emotional expressions in complex CGI scenes. Conversely, AI-driven software like deepfake technology, though controversial, has been used to create realistic face swaps in movies. For instance, it was used in The Irishman to de-age actors, offering a cost-effective alternative to traditional CGI. Additionally, AI is used in color grading and editing. IBM Watson was used to create the movie trailer for Morgan, analyzing visuals, sounds, and compositions from other movie trailers to determine what would be most appealing to audiences.
• Film Editing?: Instead of spending arduous hours culling raw footage, AI-powered software can efficiently identify and categorize sequences. This drastically reduces labor hours and allows editors to focus on what they do best: crafting compelling narratives.Key contributions of AI in film editing include: (a) Scene categorization — AI software can dutifully categorize shots based on parameters like outdoor or indoor scenes, specific character presence, or even the mood of the scene. (b) Automated assembly — After categorization, AI can compile a rough cut based on predefined editing principles, serving as a great starting point for editors. (c) Emotional timing — Using facial recognition and semantic analysis, AI can help in determining the emotional weight of a scene, guiding editors on optimal scene length for maximum audience engagement. Tools like Adobe’s Sensei use machine learning to automate tedious editing tasks, such as object removal and scene stabilization.?

Post-production and Sound Design

AI’s role in post-production is continuing to expand, making tasks more manageable and further enhancing the quality of output. It also significantly reduces the time and cost involved.?

• Special Effects and Computer Graphics: In the realm of visual effects (VFX) and computer graphics (CG), AI is pushing the boundaries of creative expression. It speeds up the process of creating complex CG characters by making intelligent decisions based on a repository of details parsed from myriads of data points. Using VFX enhancement, precise tracking and compositing, enabling artists to create more realistic and detailed special effects. Also, AI can analyze and render scenes in real-time, fostering a more flexible and interactive design process.
• Color Correction and Grading: Another area where AI shines brightly is in color correction and grading. Traditionally a time-consuming task, it can now be accomplished more efficiently and accurately with AI’s assistance. The artificial intelligence software is capable of analyzing each frame, adjusting colors and light levels accordingly to create an aesthetically pleasing visual experience.
• Sound Design : AI algorithms can create realistic soundscapes and much more such as:

1. Sound synthesis: AI algorithms can generate diverse sounds, modulating them based on input parameters, thereby raising the bar in sound design.
2. Noise suppression: The AI can detect and eliminate unwanted background noise in post-production, ensuring clarity of dialogues and sound effects.
3. Speech enhancement: AI can improve the clarity and intelligibility of speech in your mix, particularly useful in documentary or reality scenarios with challenging field recordings.

For example, Skywalker Sound, part of Lucasfilm, uses AI to categorize and retrieve sounds from their vast library, speeding up the sound design process.

Distribution and Marketing

AI's role extends to movie distribution and marketing. Platforms like Netflix use AI algorithms to suggest films to viewers based on their viewing history, enhancing the user experience and increasing viewership. In marketing, AI tools analyze social media data to gauge audience sentiment and preferences, guiding marketing strategies. For example, Warner Bros. used AI to optimize the marketing of Aquaman, leading to its box office success. Furthermore, AI-powered chatbots are used for promotional campaigns, engaging audiences in innovative ways. The chatbot for the horror movie Morgan interacted with users, creating a personalized and immersive marketing experience.

Future Potential of AI in Movie Studios

AI? is rapidly changing the film industry, from script analysis to post-production. In the future, it has the potential to revolutionize filmmaking in many more ways.

One area where AI is expected to have a major impact is virtual filmmaking. AI can help create realistic virtual environments that reduce the need for physical sets, offering filmmakers more creative freedom and reducing production costs.

AI can also be used for AI-driven narrative development. By analyzing audience preferences and societal trends, AI can help filmmakers create storylines and genres that are more likely to resonate with viewers. This could lead to more successful films that are closely aligned with audience interests.

In addition to narrative development, AI can also be used for enhancing immersive experiences in movies. AI can be used for augmented reality (AR) and virtual reality (VR) to create more engaging and interactive storytelling experiences for viewers.

Another potential application of AI in filmmaking is individualized match making of movies to specific moviegoers. AI systems can be trained to understand viewers' psychographics and neurolinguistic data. This information can be used to match viewers with movies that are likely to appeal to them, making the moviegoing experience more personalized and enjoyable. Through this combination, an AI system understands each person almost as well as a best friend does. Thus, movie studios are looking at a powerful marketing tool that can identify the specific aspects of each movie that would appeal to an individual moviegoer. Moreover, through this AI technology, studios can speak to each person, just like a best friend, to share why the person would enjoy the movie. This is a sharp contrast from today, where only certain aspects, like action, who did it, etc., are played up to the mass audience. This is why companies like 3XM Technologies are revolutionizing the industry.

In short. the future of filmmaking, shaped by AI, promises to be more efficient, creative, and closely attuned to audience preferences

Conclusion

As we’ve navigated the past, present and potential future of AI in the movie industry, it becomes evident how this fast-developing technology is poised to significantly shape the future of the cinema industry. From augmenting creativity to optimizing costs and efficiency, AI’s influence is undeniable. However, it doesn’t come without its challenges.

The use of AI in filmmaking is still not that widespread, but as technology continues to evolve, it has the potential to revolutionize the industry. By automating tasks, reducing costs, and enhancing creative possibilities, AI can help filmmakers create movies that are more entertaining, engaging, and accessible to audiences of all demographics. Movie studios that embrace these innovations are likely to stay ahead in a highly competitive and rapidly changing industry.?

Of course there are fears of job losses and ethical concerns related to AI-generated content, such as issues of authenticity, ownership rights, and potential job displacement in the film industry. It is important for filmmakers to consider these concerns while embracing the possibilities of AI. But one thing is for certain: AI is here to stay, promising a fascinating future for the industry.

The perfect climax is a blend of human genius and AI-driven technology. The cinema of tomorrow is a script that we are all writing, a movie yet to be directed, a story yet to be told.

Let’s embrace the new era of motion pictures shaped by Artificial Intelligence, together. Do engage if any of this resonates or you would like to discuss how we may work together on AI & data driven digital transformation in your business.