A conversation with Juan Delcan

A conversation with the director of the award-winning documentary Age of AI and the AI film Seeing Red

A little background. Juan is a formidable director, designer and animator.?? I’ve known Juan for many years and we’ve worked together many times in the past.? Subsequently he moved to LA and we’ve stayed in touch.

In late 2022, Juan directed the film that brought to the screen a discussion between Eric Schmidt (ex-CEO of Google), Daniel Huttenlocher, the Dean of the MIT Schwarzman College of Computing and Henry Kissinger on the future and implications of AI, entitled “Age of AI and Our Human Future”.? While the film centered around broader AI implications like robotics and future employment issues, Juan chose to bring this important conversation to life using generative AI in many of the sequences.? You can see the film here

Juan, how did this documentary come about?? The production company Sandblast brought me on. I don’t think honestly that they were aware of the AI explosion that was about to happen in 2022. The AI capabilities were literally coming to life in front of my eyes.? I found that whatever I was doing was becoming obsolete – in a matter of days!

One thing I found is that in generating images and video in AI it can look beautiful, but AI keeps offering me things that is not what I’m asking for and you have to fight it. You have to put in very specific parameters or it’s not going to work. Right now, AI is a little stubborn.

How did you work with the prompts? Well first, the order of the prompts is very important and strangely, you can write the same prompt and hour later and you’ll get a totally different result. AI is even better at certain times of the year or moments in a day!

Earlier this year Juan made a film that was created in its entirety with generative AI called Seeing Red. For this 5-minute piece, viewable on Juan’s Instagram, he generated over 100 separate shots and marked a significant step in creating original video using generative AI.? This piece is pre-Sora (which is still to be released), and what it took for Juan to create this dazzling array of images is the discussion in our hour-long interview.?

Astonishingly, this entire piece took just 3 days.? When you step back and think about achieving such a complex film any other way, the days would stretch into weeks.?

Juan, Why did you decide to make this film?? My journey started in the strangest way. I initially planned to create a film inspired by Goya’s painting “Saturn Devouring His Son.” This dark and grotesque artwork always captured my imagination. One day, as I was thinking about this project I noticed a red plastic bag dancing in the air. Intrigued, I grabbed it and started filming the bag in various scenarios, captivated by its movement and the vividness of its color.

As is often my process, I put the project to sleep for about three years. When I revisited it, I realized that with the advent of AI generative images and videos I could finally bring this project to life in a new and exciting way. While I didn’t end up using any of the original material, the core idea evolved, and I came to feel that we humans, are a bit like Saturn—consuming and being consumed. This blend of traditional artistic influence and modern technology shaped “Seeing Red” into what it is now.

What was your general approach?? Because AI is a moving target and keeps improving, my general approach was not to be too precious -? otherwise it would only be perfect in some infinite future!? So the piece was really a collection of ideas. Sometimes to get the right shot, you need maybe at least 30 tries, so you need a lot of iterations.? I think we need to understand that that in generating AI, words have value. So it’s interesting that talking again and conveying ideas is going to get to the point and be better talkers!

What AI programs did you use?? Stable Video Diffusion was very good with camera moves. Pica was the most surprising and RunwayML was the most reliable of them all.? I also used Topaz as well Also Midjourney, Gemini and Dall-E 3.

Tell me about some of the techniques you used?? I sometimes used different AI programs on the same shot.? Also, us visual artists are not necessarily the best copywriters. So I used image reference. I find a visual reference and then spend a lot of time creating the perfect reference in Photoshop – It’s almost an artform in itself.? With programs like Midjourney or Dream Studio you can give them a reference and dial up how close or far you want it from the reference image.

How much work did you do in post? Every shot has been treated -? especially color, varying the speed and removing artifacts of each shot. I used AfterFX, Photoshop and Cinema 4D.?

How do you see the future of AI in production – especially as Sora and the other applications from Google and Microsoft get released??? It’s not so much about what AI can do – which is limitless – I’m much more interested in how we digest the treatment as humans. People will initially be amazed at what they can do and even get drunk on the process and results. That’s going to reduce the visual impact, so the way to engagement is for us humans to have a say in the outcome.? As the speed of generative video AI gets faster, it will become near-realtime (audio is already there) and video will become more like a videogame for the audience.

What type of artist is going to be most effective in this area? My superpower is that I can actually convey ideas through drawings. You can even put in a prompt “I want an award-winning photograph”.? In the end what makes you effective is the collection of choices that you make. Those choices are consistent and have a specific mission and objective that is going to result in really good outcome. For the video I made, anybody could come up with these images, but it’s the way I put it together, the choice of music, the choice of color etc.? – all those choices are mine.

One thing that’s particularly beautiful working in Discord and Midjourney is you can see everybody around the world coming up with their own prompts – that can help you craft your own. It’s quite humbling.

I think a key part of every artist’s day is to allocate 20% of your time to learning.

Juan Delcan, Thank you!? For this article, I have posted it on my blog as well as on Medium.? I’ve included links to all the programs and applications discussed here. There are growing communities of AI filmmaker creators. As the new generation of applications and capabilities get released, films like the two Juan created will be markers in the rapid development of generative AI film.? How a visual artist like Juan can bend AI to their artistic will demonstrates a pathway to integrating generative AI and human creativity.

Peter Corbett

[email protected]

LINKS

·????? Age of AI and Our Human Future

·????? Seeing Red

APPLICATION

·????? Dall-E-3

·????? RunwayML

·????? Sora (pre-release) ??

·????? Stable Video Diffusion

·????? Topaz

·????? Midjourney

·????? Dream Studio

·????? Pica

·????? Gemini

·????? ChatGPT