Adventures in AI Filmmaking

The Hype, the hope and AI as a bonafide creative tool

1. How it all started

At the beginning of this year, driven by my undying curiosity, I dove head first into the world of AI. I was eager to uncover what it could offer creatives, explore its limitations, and discover practical ways to integrate it into my creative workflow. The curiosity had started at the end of 2022, AI had become a hot topic at my previous agency, and to be honest, I was still struggling to understand how much it would impact the creative industry and myself, plus FOMA was starting to take hold of me, so I decided that research and training was the best way to form an honest opinion.

Back to the future?

But first, I'd like to take you back to the late eighties when I was a young graduate entering the creative industry. At that time, hand retouching, paste-up artists, mechanical tones, sticky borders, typesetting machines, Letraset, and AGFA stat cameras were still common. However, the desktop revolution was beginning with Apple computers, printers, and scanners becoming more prevalent. Everything was still simple and in black and white, with no Photoshop yet, but Quark Xpress, MacPaint, Pagemaker, and Aldus Freehand were helping designers produce quality layouts, logos, and illustrations. For me, it was an exciting time, but many older colleagues saw it as a threat, claiming it would be back to business as usual in six months. Unfortunately, they were the first to lose their jobs. I see similar parallels now with AI; if you think it's a fad and ignore its benefits, you may be putting your future at risk.

History repeating itself

When new technology emerges, workers worry about their jobs. This has happened not only with desktop computers but also in the industrial revolution. My own family history shows five generations of silk weavers suddenly struggling to adjust in the poverty of late nineteenth century London‘s East End. New technology replaced many jobs, especially in textiles. This caused problems for many workers. Some could retrain, but not all. How might we solve this problem if AI really is a threat to our livelihoods?

The Haters

The backlash against AI can be seen in the same way and to a certain extent is rightfully justified. AI is used to create images, music, writing and videos that spread through the web, making the internet an even stranger place to navigate. There is still very little protection for intellectual property and copyright. OpenAI, Midjourney, Udio and Meta have already been sued. As cases continue, companies using generative AI should watch for guidance on AI and intellectual property. They should also check if they need risk mitigation strategies and clearer guidelines on fair use of AI-generated content. Adobe has so far avoided this by using only their own content on Firefly. One solution could be flexible licensing models for AI-generated content.

The uberfans

The hype over AI, especially after the teaser for Open AI’s Sora, has meant a glut of information about new tools on every news channel, be it traditional television, industry articles or on social media. As someone who is part of a large global AI network on LinkedIn it seems almost every day a new platform is released or an existing on adds new amazing functionality to there suite of tools. It is very inspiring but at the same time overwhelming. I’m glad I started my journey when I did, if I had started today it would have been even more confusing.

2. Where I started my journey

Initial steps

I had some previous experience in video directing for advertising, I had recently completed a month course on animation and video editing training, so I decided to continue following one of my passions and take an AI filmmaking course. In January, maybe even now, there were very few options, but I quickly found a course that was professional, very engaging and affordable. Over the course of a month, through video training and weekly homework, I went from being a complete novice to having a solid framework for creating cinematic videos. For the first time since I'd discovered my love of Apple Macs at college, I was thoroughly inspired; suddenly my only limit was my imagination and every experiment a way to improve my skills.

Tools and Technology?

Over the next six months, I’ve tested multiple AI tools for video, voice and music creation, giving each experiment the chance to create something engaging and original. Outside of practice makes perfect, knowing camera angles, lenses, depth of field, camera stock, lighting and how traditional films are made and some filmmaking jargon is very helpful to get the best possible results. If you are having problems getting the right images, then use an AI tool like ChatGPT to refine your prompts or find images you like and get Midjourney to break them down into style, lighting and techincal details. Where possible try out free tools first, many platforms such as Haiper and Luma Labs have great text-to-video and image-to-video tools that create sometimes very wild but also very compelling animations that are day by day getting better in quality and resolution.?There seems to be two camps emerging from the early adaptors to generative AI, one side is from a more creative side and one from a more technical side, along with Runway, Pika, Leonardo AI there is also platforms Stable Diffusion and a very powerful GUI called Comfy UI. My experience so far has only been in the former, but I think a good mix of both is a healthy attitude if you are looking to implement generative AI into your workflow.

3. Where I am now

I'm now at a stage where I feel comfortable entering my work in AI video competitions and suggesting it to clients for storyboarding, previs and creating video content for social media, with the caveat that it's still in an experimental stage. It's also been a lifesaver for projects with tight time and budget constraints.

Community?

Being part of a large online network on LinkedIn has also meant that I'm regularly contacted by production companies, AI agencies and start-ups and have even feel confident applying for jobs that I wouldn't have dreamed of doing before.

I've also spent the past six months talking about and being a cheerleader for generative AI to anyone who will listen. The response from my friends and colleagues in the creative, artistic and music industries has been surprisingly mixed. Filmmakers and artists are curious, willing to learn and among the early adopters, designers and art directors are still on the fence but open to learning, and musicians seem to hate AI. I was a little surprised by the latter, as musicians have been experimenting with synthesisers and drum machines since the late 1960s, perhaps with AI music companies collaborating and involving musicians, as Stevie Wonder did with the British-American electronic music duo of Malcolm Cecil and Robert Margouleff, or as Herbie Hancock embraced synthesisers when they first appeared on the music scene.

4. The case for and against?

The Benefits

There’s no argument that AI is currently overhyped. We’ve seen the embarrassing fails of Chat GPT written resume and the six fingered attempts of early text to image LLM. However as a versatile creative and to be honest a bit of a lone wolf it has solved a lot of problems and has enabled quick learning of new skills. I‘m now director, cinematographer, set & costume designer in one. From my perspective, here are a few benefits:

• A basic understanding of generative AI can only reduce misconceptions about its capabilities
• Time saved by using AI as an assistant can now be put back into critiquing our creative work
• More experimentation makes us better creatives and more versatile in our skill sets
• Learning to write better prompts, leads to us to not only a better understanding of how photographers and cinematographers work but also expands our vocabulary
• I’m more of a visual person and being an Englishman living in Germany with clients everywhere means also having to work bilingual, language writing companions such as Deepl not only help expand my language skills it speeds up writing in a foregin language

The Risks

Even though computers have been a defining part of my career, my automatic go to when I start thinking is my sketch book, regardless if it’s a logo, a film idea or breaking down a customer problem creatively. Over usage on one tool has it’s risks too:

• The risk of being too overdependent on AI is that it will make you lazy. I noticed after a while that I was doing this and that’s when the quality and originality of my work suffered
• Just taking the first result, rather than experimenting, ideating, sketching or challenging biases will result in low quality output. In other words, a poorly thought out and developed concept will still suck with or without AI
• Under appreciation or devaluation of creative and artistic skills such as conceptualisation, illustration, photography, music composition in comparison to results created with generative AI work will again lead to good but not original work, the onus here should be on educating your clients in the same way you would when either buying royalty free images or hiring a photographer for a photoshoot
• Tools such as Midjourney are excellent at recreating scenes that would be either time consuming or impossible to shoot, however faithfully recreating a scene that uses a real life example such as a specific location is currently tricky, expectation levels need to be set to avoid disappointment

5. In conclusion

For creatives, AI is our great equaliser, we now have a diverse range of skilled helpers to guide and support us, so we now have no limits or excuses not to raise the bar every time. In a world where the vast majority of design, advertising and art is either poorly thought out, cliched or just plain ugly, we now have no excuse not to change that.

My opinion: use AI, but don't stop using your imagination, the experiences you've had in your life, your personal taste and your brain is infinitely more complex and individual.

After six months reinventing myself I’m very optimistic about my career and about my creative future. You can see some of my experiments here:

https://www.youtube.com/@studiolonepine/videos

Resources:

If you are still unsure about AI but are curious to learn more, then here are a few links to get you started:

https://curiousrefuge.com/blog/ai-filmmaking-tools-for-filmmakers

https://kikiandmozart.beehiiv.com

https://www.nejcsusec.art