So, Richard Dawkins got duped by a bot believing that language models are conscious. A lot has been said about this one more episode of Lemoineism, and from a perspective of consciousness studies, Matthew Sheffield gets to the point in his article on Richard Dawkins and the Claude Delusion where he writes about how mind is not something you have, but something you do: “minds are processes continually enacted by our embodied perceptions and responses”.
In his piece in the New York Times, Leif Weatherby argues that this illusion of mind-pareidolia stems from the novelty of the cultural artifacts synthesized by AI, which directly relates to my view of LLMs as Interpolatable Archives:
The bot is a complex mathematical function performing statistical operations on data, but the output is stories, images and memes — the very stuff of culture. (...) an A.I. model doesn’t need a mind to be a serious cybersecurity threat (...) the reason it’s so striking is precisely that it doesn’t require a mind. It’s a novel form of culture.
We don’t expect meaningful and rhetorically powerful prose to come from anything but a conscious mind. But now it does. We cannot afford to believe the marketing message from A.I. companies that we may be dealing with some spiritual essence.
For this short essay, however, I’m not too interested in the question of machine consciousness, we’ll only touch on that briefly.
What I’m interested in, for now, is a point Dawkins almost gets right, when he writes that
Brains under natural selection have evolved this astonishing and elaborate faculty we call consciousness. It should confer some survival advantage. There should exist some competence which could only be possessed by a conscious being.
This is true, and every conscious human being is a proof of that, because all conscious human beings show “some competence”, arguably, some more than others.
This connects to my rejection of Emily Bender’s arguments in her parrot paper, where she claims that “Text generated by an LM is not grounded in (...) any model of the world, or any model of the reader’s state of mind”, which just isn’t true anymore, because, as Leif Weatherby lays out in his book Language Machines, “language is complex, cultural, and even poetic first, and referential, functional, and cognitive only later” and that this “poetic language is (...) computationally tractable”.
These poetics in language are picked up by the model from human practice of the linguistic form, and it learns true meaning of true things in the world in the shape of the structural vibes, and those go beyond spelling, grammar and syntax. What we have as a result is a machine that “lacks the subjective intent of a cognitive agent” which nevertheless “does encode meaning and valid semantic representations”. Is that sort of meaning weak, compared to our rich understanding of the world? Likely, yes, sure, but it’s not false.
Turns out, this sophisticated statistical analysis of big data, which picks up poetics and vibes and encodes weak-but-true meaning, is enough to produce the competence Dawkins talks about: “Are there two ways of being competent, the conscious way and the unconscious (or zombie) way?” Yes, obviously there is.
Obviously ChatGPT and Claude and Gemini and all those models of language, which do encode true meaning in the shape of vibes picked up from poetics in linguistic practice, can solve all kinds of problems, from identifying my houseplants and giving me tips for watering schedules, or exploring my thoughts and weird ideas I have in the night, or automating up to 100% of code output in whole companies. We distilled actual intelligence from statistics about linguistic practice into a model of language.
So, why is artificial intelligence not cognitive, while human intelligence is? Dawkins nearly gets there when he asks
Could it be that some life forms on Earth have evolved competence via the consciousness trick — while life on some alien planet has evolved an equivalent competence via the unconscious, zombie trick?
The short answer to this question is: No. Life as we know it can’t evolve competence without consciousness, because life and consciousness and your brain are processes, not things. Consciousness is inherently bound to a “nowness”, to the moment in time in which you experience the world. You can’t experience the world at any other point in time but the present.
I fail at imagining how any lifeform, alien or not, can decouple from that “nowness”. Life always evolves from matter which organizes itself and interacts with its environment and it always has to do that in the now. Outside of cool thrilling scifi-novels I fail at imagining life not bound to the four dimensions of our universe. I can conceive of life based on a very different chemistry, or material composition, but I can’t imagine life uncoupled from time.
Phenomenal consciousness is temporally coupled with and entangled in its interaction with the world which constantly updates your body, most obviously in the neuroplasticity in your brain. You can’t experience a red flower the exact same twice: When you look at the thing once, your brain changes in subtle ways, and when you look at it again, you look at it with a brain that was changed by looking at it in the first place. This “first place” precisely is Heraclitus in a nutshell, who wrote about why “You cannot step into the same river twice“. This change happens constantly, a million times per second, it’s a constant process of flux that’s coupled to an interaction with the outside world in the present.
I could go on and explain why this is also what life is, and how subjective experience likely evolved from movement in early organisms, but that would require an essay that easily exceedes the 14k words i just wrote on Interpolatable Archives. We’ll go there, but for now, let’s keep it simple, and return to the question of intelligence without consciousness.
Defenders of AI-consciousness, confronted with the fact that emotions and feelings in the now are essential for phenomenal consciousness, often answer with an allegory like “Airplanes cannot fly. Flying is an evolutionary mechanism that first insects and then dinosaurs used to avoid danger”. I’ve seen this argument freshly deployed in the face of two papers arguing against artificial consciousness. It’s a clever trick that misses the mark: the “flying” in question is intelligence, not subjective phenomenological experience.
I wonder if those defenders of AI-consciousness would still instist on their arguments if they had ever experienced LLMs without temperature. If Dawkins were to use a language model with no randomizer plugged in, the consciousness delusion would immediately vanish and he would realize he’s conversing with a deterministic statistical model, because it’d spit out the exact same result for the exact same prompt every single time. A language model is frozen statistics of conscious decisions put in collective human writing, a model you can navigate and interrogate with a text interface. Inference in AI is ahistorical and weights are frozen; it does not exist between prompts. Inference is discrete, and prediction happens across a static dataset where neither the data changes nor the model’s approach. But interference and prediction in life has continuity and presence, life is never not interfering and not predicting, it’s a constant process. There is nothing alive in AI, nothing that is bound to a “nowness”, no substrate updating itself in realtime. Any prompt always steps into the same river.
This is why some people argue that we shouldn’t talk about those models in terms of human cognition, in terms of “understanding” or “intelligence”, and as a defense against anthropomorphizing those interpolatable archives, i agree. The soft mind-paredolia of Dawkins may be a rather harmless example, but we’ve seen much harder cases of spiraling, and the consequential delusions resulting from seeing ghosts in machines seem rampant. But the fact remains that these models show intelligence, they do solve novel problems by interpolating models of the world. It seems that we can absolutely separate intelligence from the phenomenological process present in humans.
This means that Artificial Intelligence is not just a marketing term, but the real deal. Human intelligence requires phenomenal consciousness because human intelligence requires you to feel things about stuff, otherwise you couldn’t evaluate them. But the trick of artificial intelligence is precisely that it doesn’t require consciousness to evaluate things, it just requires vibes picked up from the poetics of collective linguistic practice, as those practices already encode conscious decisions humans made earlier and deemed valuable enough to write about. And if you encode enough of these into latent space, voilà, you get precisely that “some competence” Dawkins writes about, a true artificial intelligence without any need for phenomenal experience, an intelligence minus cognition1.
And that’s a marvelous achievement.2
GOOD INTERNET ELSEWHERE // Bluesky / Mastodon / Threads / FB / Insta
SUPPORT // Patreon / Steady / Paypal / Spreadshirt
Musicvideos have their own Newsletter: GOOD MUSIC. All killers, No fillers.
Thanks for reading.
If you’re mean, you might call this intelligence minus cognition “dumb intelligence”, precisely because it lacks the embodiment and the temporal coupling a human intelligence has, and LLMs for now sure do produce dumb things sometimes when they interpolate hallucinative outputs. I like the quasi-paradoxical sound of “dumb intelligence”, to be honest.
I can’t help but imagine what would’ve happened if OpenAI kept ChatGPT in a lab and stick to their nonprofit mission. But, i guess, the economic pressure of buying compute made necessary turning their bot into a product.
What would’ve happened if Jensen Huang, headhoncho of Nvidia, said “You know what, as a present to humanity, i gift you the necessary GPUs for free, because i was already rich before my company became the most valuable corporation in history, and we’ll do it for the benefit of humankind and for the reputational gain that comes with such philanthrophy”, akin to what Tim Berners-Lee did when he rejected capitalizing on the World Wide Web. OpenAI then could’ve done experiments in a lab, presenting us with that marvelous achievement without economic pressures, and without letting loose this cognitive experiment on all of us. But that cat is out of the bag and strolling around the neighborhood.
It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with only primary consciousness will probably have to come first.
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461, and here is a video of Jeff Krichmar talking about some of the Darwin automata, https://www.youtube.com/watch?v=J7Uh9phc1Ow
The distinction between competence and cognition is useful, and the temporal-coupling point makes the contrast much sharper than simply saying that LLMs are statistical. A frozen model can inherit meaningful regularities from conscious human practice without undergoing the continuously updating phenomenal process that produced them. But Dawkins’s evolutionary premise raises a second issue: natural selection may explain why organisms make useful self-reports, yet usefulness alone does not show that the report “subjective experience exists” is truth-tracking rather than a competent zombie output.
This video explores the meta problem of consciousness proposed by David Chalmers and argues that naturalistic processes couldn’t have given us knowledge of consciousness
This is my video: https://www.youtube.com/watch?v=WJVvZNi0Fi8
Does your account of nowness and continuous bodily updating also explain why an organism knows it is phenomenally conscious, or does it mainly explain which biological processes accompany experience? What observation would distinguish reliable phenomenal self-knowledge from an adaptive system that made the same judgments without experience?