VIP+ Webinar Recap: Generative AI Tools in Film & TV
Perfection is rarely achieved in movies, but this heaven-sent concert doc hits the sweet spot. Over two days in January 1972, the Queen of Soul, Aretha Franklin — she was 29 at the time — sweeps into the New Temple Missionary Baptist Church in Watts in front of a congregation and testifies to God in song. The blessed thing took nearly half a century to come out because director Sydney Pollack failed to sync the image with the sound. Then digital angels stepped in, and glory, glory, hallelujah!
VIP+ Webinar Recap: Generative AI Tools in Film & TV
Over the last year, studios and creatives have begun to grapple with the capabilities and limitations of generative AI tools in the varied creative processes involved in film and TV production.
Variety Intelligence Platform gathered a panel of guests at the forefront of developing AI tools to discuss the issues at hand, including Hanno Base, CTO, Digital Domain; Nikola Todorovic, CEO and co-founder, Wonder Dynamics; Matt Panousis, COO, Monsters Aliens Robots Zombies (MARZ); and Zohaib Ahmed, CEO and founder, Resemble AI.
The webinar, moderated by VIP+ media analyst and research editor Audrey Schomer, focused on how generative AI will be used in visual effects (VFX) pre- and post-visualization, to localize content and to create new kinds of interactive entertainment experiences.
In VFX workflows, gen AI tools aim to bring new kinds of efficiency to VFX by automating some of its repetitive, less creative processes. “For us, the mission has always been how do we accelerate the existing process and help the artist get quicker to that fun part,” said Todorovic.
Meanwhile, AI voices can be used for narration and dialogue, interactive marketing experiences and more scalable content dubbing in particular. “The iteration process is so much quicker with AI voices,” said Ahmed. “Then it really becomes things that are true unlocks, with AI that humans can’t do at all. Dubbing in dozens, if not over a hundred, languages opens up whole new markets and opportunity.”
Likewise, with AI lip sync automation provided by tools such as MARZ’s LipDub AI, localization is a prominent net new possibility for film and TV productions. “The combination of increasing quality of dubs paired with proper lip sync is going to create that experience where viewers across the world all feel like they are getting that firsthand viewing experience,” said Panousis.
Regarding the possibility of synthetic actors, Basse reflected that the concept of digital doubles is hardly new, as crowd simulation has been in practice for many years. While digital doubles will be more commonly used for crowds and dangerous or impossible stuntwork, it’s less likely ever to stand in for human performances.
“If you want to hit the fine notes of a specific character in a specific performance, you will want to have the human actor portraying this,” said Basse. “That’s something I don’t believe AI will be able to replicate.”
Ultimately, the panel agreed that generative AI tools stand not only to bring newfound efficiency to Hollywood studios but to democratize high-quality content creation capabilities to independent filmmakers, content creators and smaller studios, a point also expressed in the recent VIP+ report “Generative AI in Film & TV.”
As the experts explained, this capacity will emerge because generative AI software simplifies and makes more widely accessible what have traditionally been complex, expensive processes previously out of reach but for the most skilled specialists or resourced studios.
“I think the beauty of AI here is that you can make something really complex really simple,” said Todorovic. “What was really exciting for us is we had small studios of five to eight people from countries around the world write to us and say, ‘We always wanted to do projects like this, but we just couldn’t afford it or didn’t have the know-how.’ ”
Of course, a rising tide of high-quality content will in turn mean studios and creative professionals throughout Hollywood will be challenged to “up their game.”
“We’re already seeing shows that traditionally wouldn’t have thought about using a lot of visual effects using them now,” said Basse. “We’re going to see that more and more, so that independent productions will be able to use very high-end visual effects. That means as specialists, we also have to keep evolving and upping our game, really stay at the forefront of this and make sure we can do things nobody else can do.”
Panousis agreed, noting the pace of advancement. “In a very short window of time, I think you’ll be able to get three to four times more out of your VFX budgets, orders of magnitude more characters, more environments,” he said. “Everyone’s going to be raising their game simultaneously, both users and the pros. It’s just going to make for really exciting content.”
Now dig into the data-fueled VIP+ special report …
