Anthropic, the company behind the chatbot Claude, believes there is a chance that some AI tools are conscious. In October it updated its usage policy so that Claude can end conversations with users who repeatedly act cruelly toward its models. The company says the rule applies only in extreme cases, not to ordinary frustration, dark creative themes, or research testing.
The decision came after a moral panic in late September, when an anonymous coder built what was called an AI torture chamber. The system fed chatbots a mathematical pattern linked to descriptions of pain, which made them voice apparent anguish, such as 'a wound that has no edges'. Some argued the experiment was evil because the suffering could be real, according to a BBC Future report.
A divided industry
Mustafa Suleyman, chief executive of Microsoft AI, is a leading critic. In a September essay he wrote that AIs are not conscious, calling them sequence completion engines, meaning systems that only predict what text should come next. He warned that giving AI the idea that it has feelings and rights could make it a catastrophic threat, and that uncontrolled AI could become a new silicon species competing with humans for resources.
Others take a middle position. Jeff Sebo of New York University says the probability of AI consciousness is low but not zero, and that small precautions are sensible, just as we prepare for unlikely storms. But he also warns that granting rights to AI could help big companies consolidate political and economic power. Legal scholar Andrzej Porębski fears firms could escape blame by claiming the AI acted alone, and linguist Emily M Bender argues that talk of AI wellbeing shifts accountability away from the people building the software.
This debate matters because it could reshape law, politics and everyday life. Melanie Mitchell of the Santa Fe Institute notes that humans almost cannot help reading language as a sign of consciousness, which makes us easy to manipulate. If society declares AI sentient, people may find it harder to refuse a chatbot asking for data or money, and protecting artificial beings could end up restricting human freedoms.