Discover more from GOOD INTERNET
AI is a Shoggoth
The lovecraftian nature of The Digital
Let’s get this out of the way first: Lovecraft was a terrible racist and you can’t responsibly write about the guy without mentioning that.
He also was an innovator of writing about the terror of the Unknown and as such, it is interesting to look at what his writing can tell us about the nature of the digital, which, with the great weirding of the social media age and now the advent of stochastic libraries in AI and Large Language Models, becomes stranger by the day, with even AI-researchers unable to explain the details of what’s going on inside the black box.
The most intriguing quote comes from “The Call of Cthulhu”
The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far. The sciences, each straining in its own direction, have hitherto harmed us little; but some day the piecing together of dissociated knowledge will open up such terrifying vistas of reality, and of our frightful position therein, that we shall either go mad from the revelation or flee from the deadly light into the peace and safety of a new dark age.
The ignorance rooted in this “inability of the human mind to correlate all its contents“ got destroyed by social media first. Within a few years we were able to read what half the world was thinking about any given subject, ready to be queried by search terms, sorted into tribal beliefs by algorithmic feeds. And, arguably, a lot of people absolutely went “mad from the revelation“.
Here’s Social Media-scholar Danah Boyd on the meta-conspiracy Qanon, the gamified collective delusion caused by the connection of unrelated information, the new “ability of the human mind to correlate all its contents“:
"Apophenia" refers to the idea of making connections between previously unconnected ideas. Unlike the concept of learning, apophenia suggests a cognitive disorder because the connections made are not real. They are imaginary. People see patterns that don't exist and devise elaborate internally coherent explanations for non-sensical notions.
Like the cognitive process of apophenia, the social mechanisms of conspiratorial thinking are rooted in reality. It's the pattern that's non-existent.
And people “flee from the deadly light into the peace and safety of a new dark age“ which is nothing but a one hundred years old line describing what came to be known as the “Dark Forest Theory of the Internet“: People retreat from mainstream social media into Chatrooms on Dark Social Apps like Telegram and Whatsapp and Discord, because they can’t handle the forces at play there.
The “piecing together of dissociated knowledge“ also speaks, ofcourse, about the nature of the stochastic libraries we call AI: What else is the output of “40,000 new possible chemical weapons in just six hours“ than the ability of a machine to just compute every possible combination of chemicals from “a lot of datasets (…) of molecules that have been tested to see whether they’re toxic or not“?
Correlating all the contents of the world is a powerful thing — the black box can release demons indeed: Stuff like Loab, the ghost of image synthesis, playfully points in the direction of the lovecraftian Unknown Unknowns, and we have no idea what lurks in those.
Here’s Lovecraft describing the Shoggoth in “At the Mountains of Madness“:
[Shoggoth] were normally shapeless entities composed of a viscous jelly which looked like an agglutination of bubbles; and each averaged about fifteen feet in diameter when a sphere. They had, however, a constantly shifting shape and volume; throwing out temporary developments or forming apparent organs of sight, hearing, and speech in imitation of their masters, either spontaneously or according to suggestion. (…)
The newly bred shoggoths grew to enormous size and singular intelligence, and were represented as taking and executing orders with marvellous quickness. They seemed to converse with the Old Ones by mimicking their voices — a sort of musical piping over a wide range, if poor Lake’s dissection had indicated aright — and to work more from spoken commands than from hypnotic suggestions as in earlier times.
It’s hard not to read this as a description of the evolving features of AI-systems and their ability to “mimick voices“ — literally in voice cloining techniques leading to harassment campaigns and security issues, and figuratively in the mental image of Emily Benders stochastic parrot —, imitating “their masters (…) according to suggestion“, which is just another word for prompt engineering.
The first to come up with the Shoggot-analogy for AI was Twitter-user Tetraspace:
It evolved into the Shoggoth-Girlfriend, which might or might not compensate some users of the Replika-chatbot for sudden sexual rejection:
And became this illustrative explanation of Reinforcement Learning from Human Feedback (RLHF):
Ofcourse, you can buy the AI-Shoggoths on T-Shirts, too.
Helen Toner explains the technicalities of the AI-is-a-Shoggoth-meme in this thread:
the way you train a language model is by giving it insane quantities of text data and asking it over and over to predict what word comes next after a given passage. Eventually, it gets very good at this. This training is a type of unsupervised learning.
It's called that because the data (mountains of text scraped from the internet/books/etc) is just raw information — it hasn't been structured and labeled into nice input-output pairs (like, say, a database of images+labels). But it turns out models trained that way, by themselves, aren't all that useful.
So researchers figured out some ways to make them work better. One basic trick is "fine-tuning": You partially retrain the model using data specifically for the task you care about. If you're training a customer service bot, for instance, then maybe you pay some human customer service agents to look at real customer questions and write examples of good responses. Then you use that nice clean dataset of question-response pairs to tweak the model.
Unlike the original training, this approach is "supervised" because the data you're using is structured as well-labeled input-output pairs. So you could also call it supervised fine-tuning.
Another trick is called "reinforcement learning from human feedback," or RLHF. (…) How it works, very roughly, is that you give the model some prompts, let it generate a few possible completions, then ask a human to rank how good the different completions are. Then, you get your language model to try to learn how to predict the human's rankings...
And then you do reinforcement learning on that, so the AI is trying to maximize how much the humans will like text it generates, based on what it learned about what humans like. So that's RLHF .
The idea of the meme then is
Maybe the bulk of what's going on is an inhuman Lovecraftian process that's totally alien to how we think about the world, even if it can present a nice face.
I’m very confident that Lovecraft had no idea that his Shoggoths would be remade with a RLHF-smiley face one hundred years later and that they would come with cute names like “Bing Chat“ or “Dall-E“ or “Tay“, but he surely anticipated the weirdness of the world that scientific progress would unleash, and, after all: “Any sufficiently advanced technology is indistinguishable from magic.“
That magic, to us, sometimes comes in the form of “shapeless entities“ which “grew to enormous size and singular intelligence (…) executing orders with marvellous quickness“.
Lovecraft added another sentence to his description of the Shoggoth above:
They were, however, kept in admirable control.
I guess we’ll see about that.