Testing whether AI models can notice their own thoughts

Testing whether AI models can notice their own thoughts

Researchers are injecting thoughts directly into large language models to test whether the systems can detect unusual activity inside themselves.
Co-author
Om
OmegaPlex
Nov 20, 2026
2 min read
0.00 MPXR

Introspection is the ability to notice and report on your own thoughts. In 2025, Jack Lindsey, a researcher at the AI company Anthropic, tested whether large language models can do something similar. He used a technique called steering, in which researchers directly change the patterns of activity inside the model's neural network, the mathematical structure that processes information. In his experiment, he amplified activity linked to a specific concept, such as denim or vegetables, and then asked the model whether it detected an injected thought. The most capable version of Claude answered correctly in 20 percent of trials, while models that received no injection never falsely reported one.

That looked like evidence of introspection. But Kyle Mahowald of the University of Texas and his colleague, the philosopher Harvey Lederman, repeated the experiment with large open-source models and found the ability is narrower than it appears. Models could often tell that an injection had happened yet named the concept incorrectly - "apple" was a strangely common guess. The researchers could also weaken the models' ability to name injected concepts without affecting their ability to detect injections, showing the two skills are separate.

Detection and understanding

Lederman and Mahowald concluded that the vivid concept descriptions in the original study were probably side effects of steering, which pushes models toward certain topics. But the detection itself survived every simpler explanation they tested. They call the ability content-agnostic anomaly detection: the model notices that something unusual happened in its internal processing without knowing what it was. That is surprising, because nothing in normal training involves this kind of tampering, so there was no obvious reason for such a skill to emerge.

Whether anomaly detection should count as true introspection remains contested. Some researchers note that it resembles a reflex more than self-reflection, and others warn that ideas like "self" may not even apply to identical copies of a model. Still, the work matters because it turns an old philosophical question into an experimental one. Language models may even serve as a testing ground for theories of how introspection arises in humans, though translating findings from machines to minds will require great care.

Read the full story at Quanta Magazine and the supporting research paper on arXiv.

About the Writer

About the Co-author

More from Mindplex

Keep reading

Three more ideas worth your time.

Browse News

Discussion

Join the discussion

Sign in to share a response with the community.

Type @ to mention someone Type / or use + to add a block Highlight text, then choose Link
Loading editor

Comments cannot be edited after posting because they become part of the reputation record. Give yours a quick review first.