Why The Man Who Invented AI's Core Algorithm Says Consciousness Is Real | Paul Werbos

Putting aside some of the 'crazy talks, 😋 Werbos's critique of Elon Musk and Neuralink is spot-on and bravely honest, especially given how much of the tech world idolizes Musk. For me, the "Borg warning" is real, it's the logical endpoint of treating consciousness as something that can be hacked and optimized without understanding its deeper nature.

The real problem isn't technology and it cannot be solved via technology. The real problem is the tech culture! that surrounds it. True or not doesn't matter but almost all the anecdotes he share reveal a mindset that has become normalized in elite institutions. The tech bros are shity 🤷🙎

Dr. Werbos has articulated something that many of us in the AI ethics space have been dancing around but couldn't quite put into words. The IBM story about "turning people into things" is absolutely chilling, yes I know it sounds like a conspiracy nuts' quackery. However, the same thing is happening right now with how surveillance capitalism treats human beings as data points to be optimized rather than souls to be respected. What makes his perspective so valuable is that he's not some Luddite technophobe; he literally invented the mathematical framework that made modern AI possible. When someone with that technical credibility warns us about where this is heading, we need to listen.

The point about utility functions is particularly crucial and overlooked in mainstream AI discourse. Every AI system has an implicit utility function, whether we acknowledge it or not. I am a developer and I know the danger isn't just in malicious actors (which is dramatic and which is easy to identify). The danger is in the blind spots of well-intentioned engineers who don't realize they're encoding values that might not serve human flourishing.

This would 'ave been so much more watchable and entertainin if the sound quality wasn bloody awful, reelly proper terrible sound quality! Now, to the meat of it, yeah, this Werbos fella's concept of the noosphere as a "member of a species" which crisscrosses the universe is one of the most provocative ideas I've come across in years, I tell ya. It raises proper deep questions about the nature of intelligence, evolution, and our bleedin' place in the cosmos. If the fella's right, then the survival of our lot is about our relationship to a bigger intelligence system. The old geezer described it as like a kind of cosmic adolescence where we're learnin' to become part of somefink much bigger.

This is where I turn into a right miserable sod, 'cause our lot never bloody learns.

The metaphysical claims about psychic phenomena, souls, and the noosphere in this video are completely unfalsifiable and do not belong in serious scientific discourse. The fact that he had a "personal experience" in 1967 that convinced him of these things is not evidence it's anecdote! Millions of people have had experiences that they interpret as spiritual or psychic, unless it's reproducible it's not scientific. Unfortunately, each and all evidence for psi phenomena is not reproducible. The Bayesian prior for psychic phenomena is so incredibly low that extraordinary evidence is required, and it simply doesn't exist.

Furthermore, the narrative about IBM and "turning people into things" reads like paranoid conspiracy theory rather than sober analysis. IBM is a corporation, not a world government, and while their business goals may be concerning, there's no evidence of a coordinated plan to eliminate human agency. Similarly, the claim that AGI is "real now" and that Werbos "seen things in all three countries that people would kill him for talking about" is classic conspiracy rhetoric. Fruitless vague claims of secret knowledge that can't be verified or challenged. This kind of thinking undermines the very real and important conversations we need to have about AI governance by muddying the waters with unsubstantiated mysticism.

This episode was quite a surprise. First, I had never heard of Dr. Paul Werbos. Second, the historical framing of this conversation is what makes it so urgent. Dr. Werbos correctly identifies that we're not just facing a technological crisis but an evolutionary one. The pattern of "top dog species" going extinct when the ecological network changes too dramatically is well-documented in paleontology, and his point about humans being the current top dog is uncomfortably apt. What's different this time is that we're not just changing the environment, we're creating new forms of intelligence that could render us obsolete. We are creating Artificial General Intelligence aka AGI. His Cold War comparison is useful, but I'd argue the stakes are much much more higher now because the danger isn't just mutually assured destruction, but mutually assured irrelevance!

Now a days, I feel uncomfortable, almost all the time, and here I have an uncomfortable question: what happens when we don't have a von Neumann figure this time?

We are told (it's a propoganda on megaphone now) that the AI governance space is filled with people who have good intentions. Okay, let me accept their intention, but sadly these people lack the deep mathematical understanding to actually guide this technology.

Werbos might has the philosophical depth and this is why his warnings carry weight, at least for me. Here he suggested the world should start building a "better internet" that moves with humanity like Tai Chi. Though this suggestion sound poetic, the engineering details are what we really need and very urgently. We have to much to good ideas but zero technical suggestions!

I am so surprised by the fact that Werbos is offering to share those detail technical guidelines, if this man is not not one of the usual 'eccentric' old men, maybe he deserve serious attention from policymakers and tech leaders.