Some time ago in a chat I made a half-joke: current large language models (LLMs) may well be conscious, but only during training. I thought (and still think) of consciousness as some kind of sense of how one's inner mind is changing in time. In an LLM, the "inner mind" is in the weights, which are frozen and stop changing after training. Hence my half-joke.
In a recent paper titled "A Disproof of Large Language Model Consciousness: The Necessity of Continual Learning for Consciousness" (revised January 2026), Erik Hoel makes a similar case.
Hoel's argument is formal rather than intuitive. He starts from the requirement that a scientific theory of consciousness must be both falsifiable and non-trivial. Deployed LLMs are input–output equivalent to systems (a single-layer feedforward net, then a pure lookup table) for which no theory of consciousness can coherently call them conscious. That yields a disproof of consciousness for frozen, deployed models. He then shows the positive side: theories that require continual learning avoid that collapse and can still be non-trivial and falsifiable for humans. The hypothesis he draws is that, if continual learning is tied to consciousness in humans, the static character of current LLMs is tied to their lack of it. Unlocking genuine continual learning, he notes, could reopen the question.
On the half-joke specifically, Hoel is careful but explicit: the disproof "does not rule out LLM consciousness during their training in the same way it does for the deployed 'dead' artificial neural network." Training is plastic; inference is a frozen map. He does not claim models are conscious while training - only that the formal argument that kills the deployed case does not automatically kill the training case, and that this remains open. Sabine Hossenfelder popularized that distinction in a September 2026 video ("AI Could Be Conscious During Training, Not When You Use It"), framing Hoel's point as the claim that any candidate experience would sit in the weight-update phase, not in the chat session.
Ben Goertzel, founder and CEO of SingularityNET, tends to disagree. He sent me a commentary noting that frozen weights alone do not rule out consciousness during a temporary episode of in-context learning. Hoel treats the conversation history as just more input to a fixed function; Ben replies that "if anything here is a candidate for temporary experience, the stronger candidate is the interacting system as a whole, including the accumulating context." A system can still adapt its state in contextual memory while its weights stay fixed, and while that state lasts, regardless of whether that state lives in the weights or in the context, the system is not simply a static lookup table. "Calling the vector a 'weight' in one description and a 'state' in the other doesn't establish any ontological difference," he says.
I guess it all depends on the degree of permanence of the vector, a degree that still needs a clear definition. Ben adds that durable integration "needn't happen through weight updates" - "persistent external memory that is actively used and revised could also do the job." My instance of Grok Bot, for example, stores away, permanently, all that it really needs to know about me and our interactions.
This leaves a door open, without proving that any current LLM is conscious.
These are fascinating questions, but even without answering them I totally agree with Ben that there is "strong reason to regard an LLM with well-integrated, durable continual learning as a more plausible candidate for humanlike conscious experience" than a system whose new learning evaporates with its context. Continual learning may matter a lot for whether any such experience would resemble ours, without being a necessary condition for every possible form or duration of experience.
When the ice starts to melt
What my half-joke points at has a technical name: continual (or lifelong) learning. A continual learner keeps updating its inner mind as it goes, as we do, instead of being trained once and then frozen. The challenge here has a dramatic name: catastrophic forgetting. Teach a neural network something new and it tends to overwrite what it knew before. Our brains mostly avoid this. Artificial neural networks, by default, don't.
One line of research lets part of a model keep learning while it is being used. Test-time training layers turn the hidden state of a sequence model into a small model that is trained on the fly as it reads. Google's Titans adds a neural long-term memory that learns to memorize at test time, favoring what surprises it. Google's Nested Learning goes further and treats a model as a set of nested optimization problems that update at different speeds, fast parts for what just happened and slow parts for consolidated knowledge, a bit like the multiple timescales of the brain. The proof-of-concept architecture is called, fittingly, Hope.
Another line bets on sparsity. If every new fact touches every weight, interference is unavoidable; if it touches only a few, maybe not. Researchers at Meta and Berkeley showed that sparse memory finetuning, which updates only the memory slots most specific to the new knowledge, learned new facts while forgetting much less: an 11 percent drop on a standard question-answering benchmark, against 89 percent for full finetuning and 71 percent for LoRA (Low-Rank Adaptation, a common method that updates only a small set of extra parameters instead of the whole model).
Then something stranger: on November 2, researchers at MIT reported that GPT-6 Astra, asked to make the small open model Qwen3.5-0.8B better at math, read its raw weights, found a direction that caused reasoning loops, and edited 0.016 percent of the weights directly, without gradients or training. The researchers call this "Weight-Space Programming." Their paper is an early blog post, not peer reviewed, and an independent rerun suggests that the edit removes a failure mode rather than adding new skills.
How could that work? In the headline experiment, Astra was given a Python session with Qwen's weights loaded, plus some practice math problems, but no training and no gradients. It ran the model, saw that it often got stuck in long reasoning loops, traced the loop to a direction written by a few layers, and subtracted that direction from a small set of weight columns - changing only about 131,000 weights out of 0.8 billion. MATH-500 rose from 7.9 percent to 48.1 percent. Think of it less as retraining and more as reading the machine's dials and turning a few of them by hand. Still, I can't help imagining future models that rewrite their own weights, a little at a time, as they live.
None of this is yet the seamless lifelong learning that goes on in every waking moment of a human life, and Hoel warns that continual learning would be at most a necessary condition for consciousness, not a sufficient one. But perhaps the ice is starting to melt.
A body in the world
Continual learning is not the only deep difference between us humans and LLMs. We don't learn in a vacuum. We learn as bodies, and as a specific kind of body: two hands, two eyes, a sense of balance, hunger, fatigue, pain. Much of what I call my inner mind was shaped by moving that body through a world that pushes back. I don't fly and don't perceive the world by active echolocation, so what it is like to be me can only be very different from what it is like to be a bat - Thomas Nagel's famous question in his 1974 essay "What Is It Like to Be a Bat?".
An LLM has read countless descriptions of rain, but it has never been wet. Whether a body is necessary for consciousness is an open question, but a body is certainly part of what makes our consciousness the kind it is.
Robots are catching up. The workhorses of embodied AI today are vision-language-action (VLA) models, which take camera images and plain-language instructions and output motor commands. Google DeepMind's Gemini Robotics, Figure's Helix, Physical Intelligence's π0.5 and NVIDIA's GR00T are well-known examples.
A sobering note comes from the LocoManiLab testbed (October 2026). In robotics, a "policy" is the trained controller that maps what the robot sees and hears to the motor commands it sends. The best policy LocoManiLab tested drops from 80.2 percent success on familiar tasks to 25.2 percent on unseen ones. Generalization, the thing toddlers do effortlessly, remains hard. World models help by giving robots imagined worlds to practice in. DeepMind's Genie 3 generates interactive worlds from a text prompt, navigable in real time at 24 frames per second and consistent for a few minutes.
A book that touches both themes, Bin Li's “Embedded Artificial Intelligence: Principles, Platforms and Practices,” shows that Chinese researchers are very active on intelligent edge computing including robotics. The book is about putting AI into small machines: accelerator chips, lightweight networks, model compression, compilers, and cloud, local, and hybrid local-cloud deployment. It also has a chapter on lifelong deep learning. Li's preface frames the challenge with a contrast: training ChatGPT took more than 10 million watts, while the human brain fits in about 1.5 liters and runs on less than 20 watts.
Brains on a battery
That contrast is the crux of running something like a frontier model inside a humanoid robot. A robot has a fixed power budget, limited memory, and a body that must not overheat. It can't wait for a round trip to a data center to catch a falling glass. Figure's Helix splits the job between two onboard low-power embedded GPUs: a slower vision-language model that thinks at 7 to 9 Hz and a fast controller that acts at 200 Hz. Google DeepMind's Gemini Robotics On-Device runs locally on the robot, which matters when connectivity is poor and latency counts. The likely compromise is a split: reflexes and perception in the body, heavy thinking in the cloud when available. And learning on the device, rather than only running a model, is harder still, since updating weights needs more memory and energy than using them.
Our mind children
I'm optimistic. Continual learning is now a major research target, and robots keep improving. I think AIs will learn throughout their lives and will live in humanoid bodies in our world, learning from it as we do. Will they then be conscious? I don't know, and nobody does. But if Hoel is right, we'll at least be building the kind of system for which the question can be asked seriously.
I like Hans Moravec's phrase "mind children." AIs that grow up in bodies like ours, learn from our world, and carry our culture forward will be our children in a meaningful sense, and I think our successors, however they are built, will still be human in the sense that matters. And they will be better equipped than us to expand into space, carrying our story to the stars. This will be great.