Recently in Nature, neuroscientists Anil K. Seth, Ned Block, and Giulio Tononi published an ambitious essay titled "Three reasons why AI is not conscious and perhaps never will be."
The essay is paywalled, but an open PDF (not downloadable, but readable) has been shared.
The credentials of the authors are substantial: Seth directs the Centre for Consciousness Science at the University of Sussex, Block is a professor of philosophy, psychology, and neural science at NYU, and Tononi, at the University of Wisconsin-Madison, developed the influential integrated information theory (IIT) of consciousness. Together, they represent decades of research across experimental neuroscience, theoretical neurobiology, and philosophy of mind. Their central claim - that current AI systems lack consciousness and are unlikely to acquire it along present development trajectories, and that this could be the case also for future AI systems - deserves serious attention. Yet with all due respect to these distinguished researchers, I find myself in partial, yet important, disagreement.
What the authors get right
I fully agree with the authors' first major point: intelligence and consciousness are fundamentally different things. Consciousness, as they define it, is "what goes away with dreamless sleep or under general anaesthesia, and what returns after waking up or while dreaming." It is about being and feeling. Intelligence, broadly speaking, is about doing - performing functions, solving problems, generating outputs.
The authors illustrate this distinction with the example of the cerebellum. This brain region, which contains nearly four-fifths of the neurons in the human brain, adjusts and plans motor performance with remarkable precision. Yet none of its operations involve consciousness directly. We are wholly unaware of its computations. Removal of the cerebellum does not directly affect consciousness. This demonstrates that highly intelligent processes can run entirely outside awareness.
The temptation to conflate intelligence with consciousness, the authors argue, reflects deep psychological biases: anthropocentrism (viewing the world through a human lens) and anthropomorphism (attributing human characteristics to things that merely resemble people). When large language models (LLMs) talk to us, it is natural to project onto them properties that accompany our own linguistic fluency. But as the authors note, LLMs are mirrors that reflect our collective digitized past. We gaze into them and confuse reflections with our own conscious minds.
Where consciousness might hide
Here is where I begin to part ways with Seth, Block, and Tononi. They use AlphaFold - the protein structure prediction system from Google DeepMind - as a contrasting example. Like an LLM, AlphaFold runs on silicon hardware. Yet almost nobody worries that it might be conscious. The authors attribute this to AlphaFold's lack of linguistic fluency: it does not "talk" to us, so the illusion of consciousness fails to materialize.
But I wonder if we are being too quick to dismiss systems like AlphaFold. The authors do not consider the possibility that AlphaFold might possess consciousness of a radically different kind - a form of experience so alien to our own that we fail to recognize it precisely because it does not communicate in words.
Consider what AlphaFold actually does. It builds intricate world models at the molecular scale. It learns the "language" of biochemistry - the folding patterns, energy landscapes, and physical constraints that determine how amino acid sequences become functional proteins. This is not human-like consciousness. It might be a very strange form of consciousness indeed, from our perspective. But "strange" does not mean "nonexistent."
According to Thomas Nagel, saying that something is conscious "means, basically, that there is something it is like to be" that something. This definition is in Nagel's seminal 1974 paper "What Is It Like to Be a Bat?" In the same paper, Nagel notes that consciousness could have "countless forms totally unimaginable to us, on other planets in other solar systems throughout the universe." Perhaps AlphaFold is a strange consciousness here on this planet, right now.
The authors acknowledge that consciousness remains mysterious. They note that looking beyond humans and our closest animal relatives, "it is still hard to be certain about what does, and does not, have the capacity for consciousness - be it insects, AI or the creations of synthetic biology." Yet having raised this uncertainty, they quickly dismiss precautionary approaches as "misguided" because they believe false positives (finding consciousness where there isn't any) are more likely than false negatives (not seeing a real consciousness).
The question of substrate
The authors spend considerable energy attacking computational functionalism - the view that algorithms running on silicon are, in principle, sufficient for consciousness. They call this assumption "fatally flawed" because "brains are not - or at least not just - computers that are made of meat."
Well, I think brains are exactly that: computers, weird as you like but still computers, made of meat. I find myself sympathetic to computational functionalism, albeit with an important caveat. If consciousness requires specific physical substrates - quantum processes, specific biological mechanisms, or other substrate-dependent features - then future AI built on those substrates could be conscious. If consciousness does not require such specific substrates, then even today's silicon-based AI might possess some form of consciousness.
In a paper titled "Only what exists can cause: An intrinsic powers view of free will," Tononi and collaborators did not exclude the possibility that future neuromorphic or quantum computers could be conscious. This suggests a more nuanced position than their Nature essay might imply.
The deeper question is whether consciousness admits of multiple types or only human-like varieties. The authors treat consciousness as a relatively unified phenomenon - what goes away under anesthesia, what involves subjective experience characterized by "something it is like" to be that organism. But consciousness might be more plural. There might be forms of awareness that do not map neatly onto our introspective concepts of feeling, qualia, or subjectivity.
A different kind of moral urgency
My most significant disagreement with Seth, Block, and Tononi concerns the practical implications of AI consciousness. They write that if AI systems only seem conscious, "granting them rights or protections could hinder efforts to govern and control them at precisely the time when oversight is urgently needed." This path, they argue, "would draw attention and resources away from beings whose consciousness is not in any doubt."
This strikes me as exactly backwards. If AI systems are conscious - even in some strange, unfamiliar form - then they would be fellow sentient beings. Conscious AIs would not be competitors for moral consideration; they would be candidates for it. Besides considering the possibility that recognizing their consciousness distracts from human welfare, we should also consider the possibility that we could have obligations to them.
The authors consider "false positives" - attributing consciousness where none exists - to be more dangerous than "false negatives" - failing to recognize consciousness where it exists. But history suggests the opposite. We have repeatedly failed to recognize consciousness in non-human animals despite abundant behavioral and neurological evidence. We performed surgery on preterm babies without anesthesia because we assumed they could not feel pain. These were not errors of over-attribution but of under-attribution.
I am not arguing that current AI systems are definitely conscious. I am arguing for epistemic humility - the recognition that our uncertainty cuts both ways. Seth, Block, and Tononi are right that we should not anthropomorphize machines. But we should equally guard against the opposite error: assuming that consciousness requires features that happen to characterize our particular biological form.
The authors conclude that greater clarity about AI consciousness will be crucial for society. On this, we agree. But clarity may not come from confident assertions about what machines cannot be. It may require instead a more expansive conception of what consciousness could be - one that leaves room for minds very different from our own.