Interpolating Calvin & Hobbes
This is what happens when the most exclusive IP meets Image Synthesis.
If you're interested in illustration, comics and AI, there's no way you could escape the outrage over the guy who used Dall-E 3 to generate a bad Calvin and Hobbes strip. Here it is:
I cropped the screenshotted tweet to not send any more flak his way, he already had to protect his account and I’m sure people were not very friendly. But this is a bad C&H-strip. It’s a good tech demo on which he was focussing, trying to show Dall-E 3s ability to produce multi panel comics.
The algorithmic Calvin and Hobbes was not the only comic coming out after the release of Dall-E 3, there also is a Batman-comic and some other stuff i don't recognize, while this surrealist Garfield gooning around stays in the fine tradition of age old AI-generated GANfields and Avant-Garfield, not to speak of the non-algorithmic but still fan-remixed variants of Garfield Minus Garfield and the myriads of Gorefields floating around in the more interesting corners of the web.
Now, i also used image synthesis to generate some Peanuts last year. This was a time when Stable Diffusion just was released and i somewhat loved the results for their typical AI-weirdness, just look at these mutant Snoopys in space!
I loved these and other synthetic comic generations of the earlier iterations of AI-tech because of their weirdness and wrongness, but as these models get better and better now, we’re leaving the age of AI weirdness behind, which for image generation mostly means: Actually good, usable illustration that can reproduce existing IPs in a plausible way which can’t be really differentiated from it’s original source, be it Calvin & Hobbes, or Batman, or Peanuts, or Garfield.
Jim Davies himself approved of the beforementioned Garfield Minus Garfield back then and even released those strips as a self contained book. This, likely, will not happen to any algorithmic Calvin & Hobbes-comic.
Bill Watterson is famously opposed to any and all derivative stuff from his work, and fought with teeth and claw for his exclusive exploitative rights and won. To this day, he refuses to license his comics to anyone, every piece of merchandise and anything outside the strips or books is bootleged and unlicensed.
In my estimation, such a guy, who has such an incredibly protective stance on his art, would not license it to OpenAI — a company in talks to be valued at 90 billion dollars as of now and who make an image generator with which i could use a prompt like "Make Calvin and Hobbes and Trump party all night long with coke and booze" and easily interpolate Wattersons characters with whatever my mind can come up with — if his balls were on fire and a licensing deal would be the only water around. I wouldn’t be surprised if Watterson is talking to lawyers as we speak.
Fortunately for AI-artists who have no clue about Calvin and Hobbes or art and lack respect for both, OpenAI didn't care about licensing while training their commercial products, and a poor guy found out the hard way that, if you must use AI to generate comic strips, the absolute worst of all choices is Wattersons boy and his tiger. Especially when those models are getting better and better and the weirdness you can still see in the algorithmic Garfields or my Snoopy i mentioned above are about to become an artifact of the past, making place for machines which can indeed produce plausible reproductions of Intellectual Property.
Next year you will have no idea if the cool Batman on Insta which looks like an Alex Ross' was actually drawn by Alex Ross, and i think this is one more reason why OpenAI and Stability are in deep shit.
Ted Chiangs now widely read essay on how ChatGPT Is a Blurry JPEG of the Web was ridiculed by some AI people for being a bad metaphor. Emad Mostaque, CEO of Stability, has repeatedly stated that his model is a "compression of art history", just to correct himself earlier this year arguing that this comression rate is impossible, adding that "these models were never meant to be fact based".
And isn't it very "fact based" when asking an image model for an image of Batman and it gives me a Batman which absolutely looks like a Batman from DC Comics — and not just some resemblance of that, but a plausible, good, convincing Batman?
These models clearly contain a compressed knowledge about how Batman looks, and you can absolutely prompt verbatim copies of copyrighted training data given the right combination of words, which means that every image and language model does contain compressed copies of it's training data.
When AI companies now go on and RFHL the shit out of these AIs to surpress the output of verbatim Batmen or Harry Potter-books doesn't mean much, as researchers have proven that "any behavior that has a finite probability of being exhibited by the model, there exist prompts that can trigger the model into outputting this behavior". Behaviours for Image Models include all the myriads of interpolated Batmen and Calvins and Hobbeses plus their originals.
All of this is not to say that i’m opposed to all IP-theft or fan art, on the contrary. Comics have a rich history of being remixed, especially in underground zines. I own a one colored unofficial version of the Hate Smurf comic which a friend bought at the Angoulême comics festival in France, and the City Strips, collages of panels from copyrighted comics to make new architectural art, and one of my favorite illustrators is one Ermsy from Paris who uses all kinds of IP from comics to produce the trippiest pieces of popculture art i’ve ever seen. And yes, ofcourse, there’s weird ass Calvin & Hobbes remix art out there:
Many people defend OpenAI and Stability by comparing training an algorithm on licensed works to humans looking at art. But OpenAI is not a human, and their algorithms are not creative. When OpenAI and Stability released commercial products based on AI research, which in their latest iterations produce more and more good-enough ripoffs of existing IP, they made a potentially fatal mistake.
As i said in my piece on the image synthesis lawsuits:
The only thing that matters here is if the companies involved used unlicensed artworks in the creation of a commercial tool, and if the Fair Use defense applies to a machine that can mimic artistic style.
They did, and we'll see.
None of this diminishes the value of AI research. It is mindblowing that AI can reproduce illustration to such a degree that it is soon able to compete with humans, and the results have their own pitfalls, their own aesthetics, their own merrits, and merely by existing, they shine a light on exploitative practice in capitalism providing a new way to think about the world we live in.
But releasing the database of weights produced in this research as commercial tools might prove a death knell for many image synthesis platforms as we know them — and these companies can absolutely be forced by court to wipe out everything. Mindlessly opening these tech demos to the public because of some AI race may prove a dire mistake in the long run.
It is kind of funny that especially Calvin & Hobbes by the highly reclusive Bill Watterson, of all things, drives home this point.
Great take! Here's hoping for Mr Watterson to join the fray.
Here's my take on our current situation, with a clear nudge in his general direction.
https://johancb.substack.com/p/altman-and-swooper-comic-sample