Lights, Camera, Algorithm!

By now, it’s unlikely that you haven't seen the endless AI-generated Studio Ghibli portraits. It's ironic that Hayao Miyazaki himself famously said, “I strongly feel that this is an insult to life itself…” during a presentation of an AI animation project.

As Artificial Intelligence tools become more sophisticated and sharp, they offer filmmakers new avenues for creativity, efficiency, and storytelling. However, this technological evolution also raises questions about authenticity, ethics, and the future role of human creation in cinema.

In just a few short years, AI has leapt from behind-the-scenes automation into the heart of creative production. What began as a few tools for image enhancement and basic content generation has sprouted numerous startups that now offer increasingly sophisticated platforms capable of producing images, music, writing, and even cinematic-quality video with minimal human input. Using platforms like Midjourney, Sora, and Runway, creatives can increasingly collaborate with algorithms to ideate, iterate, and execute their ideas. This rapid evolution is accelerating the work process and reshaping the way we define authorship, originality, and artistic labor. The result is a seismic shift that challenges traditional filmmaking while empowering a new generation of tech-savvy storytellers.

Currently, the main players of AI offer a variety of capabilities, some with more or less overlap, but each with their own unique “look.” OpenAI (Sora) offers text-to-video generation that has the potential for rapid ideation, storyboarding, and is improving in terms of realism and scene length. Runway offers real-time video editing and generative video to expand or contract scenes and shots, and has improved massively between Gen-1 and Gen-2. The VFX team on Everything Everywhere All At Once used Runway to speed up their editing process. Synthesia, Hour One, and Rephrase.ai specialize in AI-generated avatars and synthetic presenters for dialogue scenes, which is particularly useful for commercial and explainer videos. They create realistic human-like characters, streamlining the production process for various content types that are less complex than traditional film offerings. Wonder Studio (Wonder Dynamics) specializes in replacing AI characters in live-action footage, automating motion tracking, and integrating CG characters.

AI is also changing the filmmaking workflow from pre-production to post. Dall-E or Midjourney offer specific integrations for storyboarding, and platforms like Sudowrite and ChatGPT make it easy to ‘collaborate’ with AI for scriptwriting. In post-production, AI can streamline editing, visual effects, and localization. For example, AI-powered tools can resync lips with voiceovers in different languages and enhance the quality of the footage, a tool similar to the one used on the Oscar-winning film The Brutalist. This sparked extreme controversy in the film world on the ethical use of AI, as the film used AI to improve upon the actors' Hungarian accents and vocal inflections.

All of these tools and platforms are rapidly improving. As the saying goes, “Today’s AI is the worst AI you will ever use.” It’s more important now than ever that the industry begin grappling meaningfully with the challenges and ethical dilemmas that AI presents, rather than just succumbing to its advancements. These include the potential for increasingly sophisticated deepfakes, copyright and intellectual property ownership issues, as well as the inevitable displacement of workers and labor disputes that will arise as studios become increasingly dependent on AI as it seeps into every aspect of content creation.

For those working in or interested in the industry, the question is whether these tools create the opportunity for multi-faceted creative outputs or whether AI is used in the pursuit of profit, and as such, will completely disrupt the traditional industry. Think of it this way: For something as simple as a fashion shoot, you would rely on anywhere from a minimum of three people to as many as a dozen, depending on the complexity. With the tools available today, that can be essentially reduced to one.

The other glaring question this begs is if AI will create the same level of disruption for the entertainment industry as streaming has? Imagine hundreds of creators using AI to produce films of all kinds that flood the marketplace. And most importantly, is this a good thing or a bad thing? Social media has launched hundreds of musicians and artists out of obscurity to success. It would be simple to imagine that AI could do the same for filmmakers using these rapidly evolving platforms, but does an excess of talent and creativity, peddled by an automated machine, really reflect what we want to see in the filmmaking world? Not only does the very idea of AI leave a bitter taste in the mouth, but part of the beauty of filmmaking is its process, and how hard one must work to push their project towards perfection, an achievement that is, for the most part, impossible. 

To thrive in this new environment, professionals and aspiring creatives need to focus on a hybrid strategy that combines core cinematic vision with a firm grasp of emerging technologies. As much as AI in the art world is conceptually detestable, it may be better to embrace it, rather than be left behind in the past. By taking the point of view that AI can enhance a creative vision rather than replace it, leveraging these tools for art direction, script development, and more can expand creative possibilities. But, like your first experiments in creation or writing, you will need to understand and learn how to harness these new tools without letting them consume your craft, especially since they continue to evolve at a rapid pace. Prompt engineering, style references, and creating workflows that integrate various platforms present a new learning curve with unique challenges. However, storytelling, direction, art direction, sound design, and cinematography are, so far, mainly the power of humans. They are the core of what makes filmmaking an art, and are what I find to be under threat by AI as it continues to improve. Pairing traditional filmmaking craft with modern tools will be a superpower in the (not-so-distant) future, and having the foresight to understand this before it floods the industry may just be the key to navigating a film industry we cannot yet imagine.

The Art Newsletter
Creativity. Culture. Community.

Every Thursday, we send a carefully curated drop of stories, tools, and creative insight for the next generation of artists, musicians, filmmakers, designers, and thinkers.

From cultural commentary to personal reflections, viral trends to overlooked gems. We cover what’s happening and what matters. Want local updates too? Join our Austin list for events, meetups, and opportunities.

Join The Art Newsletter