A selection of fine AI-cinema experiments

AI does South Park, The Great Catspy, A Scanner Darkly, Ingmar Bergmans Bloodborne, The Car, meta-shortfilms, experiments and commercials for fake lawyers, imaginary retrotech, cheeseburgers and beer.

May 9, 2023

I like to compare the current state of AI-cinema with the state of movies at the end of the 19th century and Eadweard Muybridges Horse In Motion, which was featured prominently as an iconic piece of film history in Jordan Peeles excellent scifi-mystery Nope.

AI cinema is similarly at a stage where we got the images running in motion, but don’t know what it really is. Back then, people who watched the Lumières L'Arrivée d'un train en gare de La Ciotat were convinced that a train would jump out of the screen and run them over. Today, we look at a very weird AI-Will Smith gobbling up Pasta and don’t know what to say.

Runway are state-of-the art for AI-video and released their Gen2 text-to-video-model a month ago. While the results are technically impressive, we are with AI-video roughly in the same spot as with the release of Dall-E 1 two and a half years ago. (Also: 2,5 years for generative AI to go from an obscure and weird glitchy computer-vision-art thing and “tech only known to some nerds“, to “we’re all gonna die, fly airstrikes on rogue datacenters“ is quite an accomplishment in it’s own).

One could argue that we’d see the same improvements on an accelerated timeframe and The Russo Brothers, directors of a bunch of Marvel-movies, in a recent interview already claimed that actual AI-cinema is less than two years away. I think that’s delusional.

Full synthetic movies are at least a decade away, if you define synthetic movies as completely machine generated with humans barely touching them, because video, just like synthetic music, adds a lot of complexity compared to pure images synthesis. Movies need to adhere not only to something like a script/prompt, but also to a story arc that is interesting and features some rythmic dramaturgy with quiet parts and climaxes, not to speak of consistency of characters and visuals on a high quality level. For videos longer than a few seconds these are not easy feats, and it will take a good while to get there.

But for short sequences you can throw together in a video app with some AI-voiceover, it’s already good enough to blow the minds of unsuspecting Marvel-directors — and mine.

Besides from everything Balenciaga, the pizza commercial for Pepperoni Hot Spot was one of the first vids from this latest generation of AI-videos to go viral, and since then, tons of clips popped up everywhere, and some are even pretty good in their weird, mutant AI-way.

So, here’s a somewhat non-random selection of good, new RunwayML-clips as a reference point for the state of AI-cinema and video synthesis. A few of these examples were made by Roope Rainisto about whom i’ve written a bunch of times already and who still delivers some of the best AI-art out there. Others come from Paul Trillo, ChristianF and Ben Nash.

Clips i wish i could show here on the ‘stack but can’t because Elon Musk has no balls:

I tried to only feature video synthesis from RunwayML whichs singular clips were not edited with Photoshop or After Effect, and divided this selection roughly into shortfilms, movies, commercials and experiments.

If there’s an awesome clip or experiment i missed, feel free to add your stuff in the comments below. Enjoy!