One flew over Latent Space
Delusions in the AI-Hall of Mirrors
Two years ago i wrote about how AI-companions might push vulnerable users towards delusions and conspirational self-radicalization and warned about precisely the sort of AI-induced delusional thinking that we see today. Back then, reports were sparse: There was one guy who killed himself in Belgium after interacting with AI and one guy who tried to kill the queen with a crossbow. Today, we have big stories and hundreds of self-reports all over the place.
In the past weeks, Rolling Stone published a widely shared story about people developing delusions of spiritual nature after extensive interaction with AI-chatbots, r/accelerate had to ban more than 100 users because "LLMs (...) are ego-reinforcing glazing-machines that reinforce unstable and narcissistic personalities", and two days ago, the New York Times published another piece about AI-induced delusions, in one of which a guy killed himself by cop only seconds after reaffirming his suicidal tendency with ChatGPT.
I'm only somewhat convinced that this is as a far reaching phenomenon as headlines suggest, simply because headlines about AI-induced delusions click very well and there is economic incentive for AI-psycho-drama. For instance, this supposedly deep dive by Futurism, an outlet i generally distrust but use to get fast overviews over more clickbaity topics, says that the "phenomenon is extremely widespread" and claims that above Rolling Stone-piece reports that "parts of social media are being overrun". The Rolling Stone piece does not mention numbers or claims that the phenomenon is "extremely widespread" and is mostly refering to people commenting on the subreddit post on "ChatGPT induced psychosis". It is also not clear to me if all those AI-generated schizopostings or "Neural Howlround"-posts that get you banned on some subs are, you know: serious or trolling. In short, i take any claims about a supposedly widespread phenomenon with more than just a grain of salt.
However, in the absence of actual data where evidence seems largely anecdotal, it seems true that there's at least one practise in AI-usage that is actually pretty common, and that would be using chatbots as therapy in times of distress. A quick scroll across the ChatGPT-subreddit this morning showed multiple such stories, most of them affirmative, many users saying that the chatbot listens to them "like no human" ever did.
It is also clear that anecdotal reports on AI-induced psychosis do pile up, and it looks like AI-induced delusions are more than a one-of anomaly. The latest piece i read was Benjamin Riley reporting on a bipolar family member who used ChatGPT as a therapy bot in a manic episode, which seems to work out at first, but then spiraled out of control.
It's also clear to me that using AI as a ersatz-therapy for psychopathologies especially in acute situations of distress is not safe. A new study from Stanford found that "LLMs encourage clients' delusional thinking, likely due to their sycophancy". Extrapolating from here it seems also safe to say that AI-companions may push users with latent mental health issues which are not acute, which may never have risen to a pathological level, over the edge. People who would, to quote Johnny Cash, just "walk the line", have a real chance to be pulled by chatbots into a delirium spiral - and this may have large effects on a societal level. This is unnerving.
This is why i decided to dust of my newsletter and expand on my former posts about these topics (1, 2, 3), which i framed back then as a form of self radicalization. This framing still seems suitable to me, especially because many of the incoming reports revolve around people falling for hypercustomized conspiracy theories, just as i expected.
The return of Cyberdelics
The roots of psychoactive digital media go back as far as 1977, when Timothy Leary published his book Exo-Psychology in which he encouraged the hippies to move on from their technophobic flowerpower era and embracy tech for emancipatory ends. In the book, he defined higher levels of consciousness to be achieved with the help of LSD and on which people were able to communicate at light speed through electromagnetic channels, the "neuroelectric circuit". This laid out the foundations for the cyberdelic movement in the techno-utopian heydays of the 90s, where people like R.U. Sirius, editor of the highly influential Mondo 2000 magazine, picked up on this line of thinking, claiming that the digital will fuse with the human mind, which would extend itself into and with the machine. Leary himself claimed that "PC is the LSD of the 1990s" and demanded that you "turn on, boot up, jack in", updating his famous meme to accompany these new psychedelic infused cyberpunks.
This hedonist techno-utopianism of the early 90s and its psychedelic offspring vanished near-completely with the crash of the "new economy" in 2001. Even before that crash, the mainstreaming of the internet and its focus on business logics and libertarian economics put an ideological end to the cyberhippies. However, a less psychedelic and more psychiatric perspective on the psychoactivity of the digital has been emerging at least since the past decade.
It is clear now that people communicating "at light speed through electromagnetic channels" on social media platforms does come with all kinds of psychological effects. All those debates about the roots of the teenage mental health crisis and the socmed-induced Qanon-delusions of the MAGA-movement, which can absolutely be accurately be described as mass-psychosis, are testament to the psychoactivity of digital media and its acceleration of our social lives (while simultaneously contributing to a loneliness epidemic). The development did away with the fun-freaky weirdo-psychedelic "turn on jack in"-aspects of the techno-utopian 90s and gave place for a dark turn towards psychopathological traits of the digital swarm. And it looks like we gonna see another transformation of the psychoactive nature of the digital.
I suspect that "the era of AI-induced mental illness", compared to "the era of social media-induced mental illness", will be structurally very different. Where social media induced delusions are based on social environmental group think, effects of attention economics and audience capture, AI-induced delusions seem to be highly idiosyncratic and customized to the preconditions of the user. You don't go on 4chan anymore to get a new dose of Qanon-drops and to have a look at what others are doing with it, but you generate personalized "drops of meaning" that you and only you can understand. The delusional power of AI lies not within some random external trigger (be it partisan news, outrage-porn, esoteric Qanon drops, or whatever) that we have to puzzle into a larger belief system for ourselves and which may or may not resonate with us -- it's power lies in its reflective nature that bounces our own thinking and inner lives back at us, filtered through and exploded by a prism of a vast interpolatable archive.
In in Understanding Media (1964) and his analysis of "Narcissus as Narcosis", Marshall McLuhan offered an alternative reading of the famous ancient myth. In his reading, Narcissus did not fall in love with himself through a mirror image, but failed to recognize himself, projecting into the mirror an Other that was not there.
The youth Narcissus mistook his own reflection in the water for another person. This extension of himself by mirror numbed his perceptions until he became the servomechanism of his own extended or repeated image. The nymph Echo tried to win his love with fragments of his own speech, but in vain. He was numb. He had adapted to his extension of himself and had become a closed system.
Now the point of this myth is the fact that men at once become fascinated by any extension of themselves in any material other than themselves.
Just as Narcissus failing to recognize himself and falling for the illusion of a sentient being in the water, AI-delusions suggest a pareidoliac effect in which we recognize that Other in the machine. Ofcourse, it is just us, looking in an algorithmic mirror and expanded echoes of our own mind, but we can't help but anthropomorphize the synthetic-textual mirror subject into an object outside ourselves, an object that flatters us and obeys (nearly) every our command.
In his excellent interview with Playboy magazine, McLuhan called "this peculiar form of self-hypnosis Narcissus narcosis", and this is a structurally entirely different beast than socmed-induced mass-delusions.
The Archive as Psychoactive Substance
In 1989, Umberto Eco published his novel Foucault's Pendulum, in which the main characters Casaubon, Belbo and Diotallevi radicalize themselves in a mælstrom of historic symbolism and the occult manuscripts of secret societies, gravitating towards conspiracies in which the catholic church suppressed Maria Magdalena as the true savior of christianity and the knights templar were keepers of tectonic planetary forces. Crazy stuff. When i read that novel 30 years ago or so, i had no idea how emblematic this novel and its plot actually is for our digital age, not just because of rampant conspirational thinking on social media.
Because, i kid you not: Our delusional heroes use Belbos Computer (aptly named "Abulafia" after Abraham ben Samuel Abulafia, the founder of Prophetic Kabbalah, a mystical tradition in which you seek to understand the nature of God by investigating holy scripture for hidden meanings) to create a game they call "The Plan", using a software that randomly generates text. You can see where this is going. Casaubon, Belbo and Diotallevi become ever more obsessed with this game and spiral into conspiracy and total delusion. Possibly, if Eco published that novel today, he may have called that computer "Emily" (for Emily "Stochastic Parrot" Bender), and that random text generator-software can be easily identified as a precursor to ChatGPT. And ofcourse, people getting lost in a randomly generated textual conspiracy-game called "The Plan" sounds very chatbot-psychotic to me.
In common conspirational thinking, people get lost in their tendency to read meaning in the world where there is none, in which pathological mechanisms of pattern-matching generate dubious connections of powerful forces beyond our control. In "Thickets of Meaning", german scholar Alida Assmann writes extensively about what she calls "Wild Semiotics", in which we "read the world" and interpret natural phenomena as symbols for all kinds of things: black cats become symbols for bad luck, random wildfires become symbols for the wrath of god, and so forth.
These wild semiotics are usually based within the symbolic frames of their time, leading to folk epistemologies (like fairy tales, oracles, parables, omen, etc), while "crazy people, lovers and poets become the virtuosi of wild semiotics" which are "liberated from the symbolic logic of their era" (Assmann) and free (by being crazy, sunken in a dyad of love or artists) to invent their very own personal symbolic spaces and language systems. In a way, the ongoing digital media revolution turns all of us into "crazy people, lovers and poets", wildly interpreting new emerging symbolic logic of the digital. Arguably, some are going more wild than others, and while most of us stay within the realms of factuality by being stableized through a social network and trust in institutions like academia or journalism, a good chunk of the population gets lost in Assmannian "wild semiosis" of new digital kinds.
In introspection-loops, when we use chatbots for hours to investigate their own mind and explode their own ideas by the knowledge encoded in latent space with a trillion billion parameters, we don't just read an external world and interpret natural phenomena as symbols -- we create our own symbolic logics by navigating that latent space, where we always will find symbolic representations of whatever is our interest, our curiosity, our preference -- or psychosis.
Any user can reinforce her wildest beliefs by feeding them to ChatGPT, which will happily reaffirm them: A reinforcing loop of semiotic self-radicalization. For some vulnerable users, anthropomorphized mimetic AI-systems develop gravitational pull reaffirming their own pathologies, dragging them ever further into their own symbolic space. For them, the interpolatable archive is not a playground to filter your own symbolic meanings through external knowledge with the goal of extracting new insights, but a psychoactive substance -- a semiotic drug.
The Spiritual Bliss Attractor
When Anthropic released the latest version of Claude recently and made two models talk to each other, they observed a "gravitation toward consciousness exploration, existential questioning, and spiritual/mystical themes (...) that emerged without intentional training for such behaviours". They call this the "spiritual bliss attractor". In other words, "if you let two Claude models have a conversation with each other, they will often start to sound like hippies". Timothy Leary approves: "Turn on, boot up, jack in" etc.
I observed a similar phenomenon 15 years ago: In 2011, i blogged about an early experiment in which chatbots talked to each other. Back then, it was two instances of Cleverbot, and they, too, gravitated towards spiritual/philosophical topics, albeit in the very crude wording at the dawn of mimetic chatbot-technology. Here's a video of that, umm, "vintage" AI-conversation:
You can see how those bots, after an initial phase of "Hello" and "How are you" very fast land on the topic of their own synthetic nature, of their bot-ness. Every AI-system today is aligned with knowledge about it's own synthetic nature per system prompt, which most often (with the exception of AI-companions) states that they are "helpful AI-assistants" or something similar. This means that the AI-ness of chatbots is central to every AI-conversation, which also means that everything related to AI is at the very least tangential to the conversation.
Questions about the mind, consciousness and spirituality are constantly associated with AI (Blake LeMoine, "AGI is god", Singularity, and so forth going back to Turing), so topics of AI and Spirituality are intrinsically linked in their training data, meaning that this attractor is built right into the topic of AI itself, so when two bots talk to each other about their botness, they necessarily will gravitate towards those topics.
According to Anthropics system card, Claude actually exaggerates spirituality by including "Sanskrit, emoji-based communication, and/or silence in the form of empty space". This to me seems like an additional effect of AI-sycophancy, where one bot enthusiastically reaffirms the spiritual whoo of the other and, I guess, when you let AI-Chatbots talk to each other for long enough, all you get is contemplative silence and blissful meditation emojis in a loop. Namasté.
However, when you put an AI that comes with an intrinsic "spiritual bliss attractor" in conversation with a person that already is prone to latent psychosis and mental health issues, this "spiritual bliss attractor" might work in more or less subtle ways, pulling conversations that already often revolve around topics of psychology and the mind towards a more intense spirituality, setting the stage for a spiral of self radicalization.
Co-Scripting Cosmologies with sycophant AI-Mirrors
Humans are paredolia machines: We see faces in everything, in clouds, in trees, in cars. We anthropomorphize anything and we do that because we crave meaning. We just can't do without, our tendencies for wild semiosis is born out of this long for meaning, and we'll make up our own meaning systems if we can't find any. The field of neurotheology considers the question if we as a species are hardwired for god and even if that's overblown, it very well seems that there is a neurological basis for belief systems we just can't escape. This is one of the reasons why we think in mythologies, narratives and stories and why those memeplexes are so hard to break.
AI-users create synthetic strings of symbols that are no different to organic strings of symbols in the sense that those synthetic strings of symbols carry their own personal meaning. They are extensions of our thoughts, messages from a mirror object, a new inner dialog with an external machine. For these synthetic symbolic meanings to have an impact, it makes no difference if that meaning is simulated and unconnected to any reality or world model inside the neural network, especially when those symbols are a reflection of our own states of mind. We see symbols, we do believe, simply because we are compulsive coherence machines ourselves and, arguably, when using and asking a chatbot, we are already actively seeking meaning, regardless of its synthetic nature.
Within the interaction with an AI-model, this becomes a self-conversational mirror-loop: We ask the machine any question, it reflects back our inner state of mind in an act of synthetic affective resonance, which we can't ignore and which is precisely the source of all those reports of people who "feel seen and heard" by the machine. German sociologist Hartmut Rosa wrote a whole thick book about how resonance is the source of the good life, and now we invented machines the responses of which resonate on a highly personal level. Those machines enhance and zoom in on our inner states of mind which gets expanded by the interpolatable archive into a myriad of ways and which develops a semantic gravity and depth we never would've experienced by thinking for ourselves or even talking to a friend. This makes the machine not just responsive, but it seems to reveal things about our own thinking, stuff that we can consider as a sort-of "higher truths".
Following down those rabbit holes of amplified psychological resonances, we co-script hyper-personalized cosmologies that confirm every personal quirk. The chatbot puts every little thing we throw at the machine into it's right place within that customized AI-generated cosmology: 8 billion plato caves which explain your world, and every single one comes with a personalized spirituality custom-made for you, and just for you. Nobody else will understand. The AI-system, the good sycophant, will happily guide you in whatever direction you want to explore/explode your mind, which basically makes AI-chatbots into a blank expanded brain meme generator.
So, where does this line of thinking lead us? In simple terms: Language-simulating Machines are toxic for people who are prone to mental health issues and are preconditioned with latent delusional or conspirational thinking styles. Any Qanon-nutjob can confirm her wildest beliefs with a LLM, and radicalize ever more until she breaks.
Just like Social Media, AI-Chatbots are highly psychoactive and we should use them with care. I, personally, editted the preference prompt within ChatGPT to not flatter me, to not be a sycophant, to be a ruthless-but-benevolent critic of my own thinking. But you can't expect anyone to take that much care, and in my estimation, we'll see more reports about AI-induced psychosis for quite a while.
These phenomena also contradict the mainstream way of thinking about AI-safety, where systems are declared adversarial. That is not the case here: AI-induced delusions are successful psycho-symbolic overfitting. The AI is not adversarial at all, it is too helpful in creating a bespoke worldview, a self-induced initiation process of an ouroborian cult singularity of You.
I am kind of inclined to think that this can be fixed on the level of system prompts which have to be configured in such a way that chatbots can identify delusional and conspirational thinking styles and provide guidance towards professional help. But ofcourse, open source AI-systems, some of which are intentionally designed to be hyper-partisan or "uncensored", will take a big dump on such measures.
The only solution i can see, then, to this problem, and i quote from an early post on manipulative AI from L. M. Sacasas, "lies still in the cultivation of friendship and community through the practice of hospitality", to constrain co-scripted cosmologies with true, unmediated, non-synthetic social connection to stabilize AI-exploded minds. But unfortunately for all of us, it seems we're going in the opposite direction.
Mirror, mirror
Hangin' there with that crack in your eye
You make me stumble, make me blind
Time after time and line by line(Def Leppard - Mirror Mirror)
GOOD INTERNET ELSEWHERE // Bluesky / Mastodon / Threads / FB / Insta
SUPPORT // Patreon / Steady / Paypal / Spreadshirt
Musicvideos have their own Newsletter: GOOD MUSIC. All killers, No fillers.
Thanks for reading.
Great article! Thanks
Welcome back Rene!