It increasingly seems to me that the AI doomers and the Big Tech companies are now either conspiring, or else at minimum collaborating and coordinating, to bring about a quite dangerous regulatory capture of the whole AI industry.

If you’re not familiar with the term, the basic idea is this:

• First, you have government regulations that were put in place, at least ostensibly, to protect people or otherwise do some good in the world.
• Then certain powerful parties—usually big companies—get so much influence over the regulators, and over what the regulations say, that the main function of the regulations turns out to be protecting an oligopoly of those companies.

We’ve seen this all over the world in various different industries. In the US we’ve seen it very clearly in finance and in medicine, and now there’s a serious effort under way to bring the same thing about for AI.

Here is how it’s unfolding with AI.

• The public fight between the doomers and Big Tech gets louder and better financed all the time.One side says the big AI companies are building something that could kill everyone.The other says this technology is going to transform civilization and has to keep advancing.
  • One side says the big AI companies are building something that could kill everyone.
  • The other says this technology is going to transform civilization and has to keep advancing.
• The compromise taking shape between “stop” and “go” becomes “go, under supervision”—and supervision needs rules, experts, approved evaluators and licensed builders.
  • The organizations in a position to supply all of those are—lo and behold— understood to be the frontier labs themselves, and the safety funders and institutes already working with them. (Never mind, say, actually independent industry organizations or whatever…)
• So a debate that started out being about whether the frontier labs should be allowed to continue turns into a negotiation over how to keep the frontier in their hands. Everybody else—open-source developers, university labs, decentralized networks, those of us building AGI on quite different architectures—is out of the picture before the discussion starts.
  • Everybody else—open-source developers, university labs, decentralized networks, those of us building AGI on quite different architectures—is out of the picture before the discussion starts.
• And the media helps this along by playing the whole thing as a ping-pong match: will Big Tech kill us all, or will Big Tech save us all? After enough back and forth, people think that’s the only game there is.

We already know what this kind of arrangement does, because here in the US we’ve run this social experiment twice at scale, i.e. in both finance and medicine… and in these cases, of course, the gatekeeping failed to deliver even the safety it was sold on. The 2008 financial crash ran on securities with triple-A ratings from the officially recognized agencies, and the opioid epidemic began with FDA-approved painkillers—and at the same time, in both cases, promising alternatives from outside the system, like cryptocurrency and traditional medicine, have been kept at the margins.

Is the current push for regulatory capture in US AI a conspiracy? For a while I had myself convinced it must be—and I still suspect there’s some element of that, though I doubt it will be provable until years after the fact. But the more I thought about it, the more I realized it basically doesn’t matter. Whether the “AI regulatory capture” course of activity was hatched in somebody’s hot tub in Palo Alto, or it’s just what emerges when everybody does what their incentives tell them to do, in the end it’s pushing toward the same thing: a regime in which only a few big companies are allowed to build advanced AI.

The part that gets the least attention is that this would be bad for safety, on top of being bad for freedom and for the distribution of wealth. What Big Tech is building now are systems with no moral agency—because they have no real agency at all—wrapped in brittle and only-partly-effective guardrails. A regime that locks the whole field into putting stricter guardrails on that kind of system would massively slow down research on fundamentally safer ways of building AGI—minds with moral agency of their own—most of which is going to come from outside the big labs, but which can be accelerated by frontier LLMs as research assistance.

Meanwhile, everyone—including those making public noise about it—knows a global pause or slowdown regarding advanced AI is not going to happen. What the campaign for one will get us is selective restriction, which changes who builds advanced AI much more than whether it gets worked on intensively

The good news is that none of the AGI future is written in detail yet. We’re moving toward regulatory capture of AI rapidly, but not with infinite speed, and not inexorably. We have a window.

Cosmists, Terrans, and who gets to build

Some modern anthropological terminology, first of all…

When I say “doomers” I mean the catastrophic-risk crowd that advocates sweeping restrictions on advanced AI: pause AI, unplug AI. I don’t mean everybody working on AI safety, and I don’t mean every last person in the effective-altruism community. (Obviously, “Big Tech” isn’t one borg mind either—much as it seems so now and then…)

I’m also going to use the word “Cosmist,” in the sense Hugo de Garis gave it some twenty years ago in his book The Artilect War. Hugo predicted that the dominant political question of this century would be whether or not humanity should build “artilects”—artificial intellects, machines with intelligence vastly beyond the human level. He called the people in favor Cosmists, because they take the long, cosmic view on which creating such minds is something like humanity’s destiny, the next step in the evolution of intelligence in the universe. He called the people opposed Terrans, because their loyalty is to the Earth and its human inhabitants, and they think the risk that such minds would do away with us is too great to run. Hugo expected the disagreement between these two groups to end in a catastrophic war before the century was out. Whatever you make of that forecast, the split he described is now the main fault line in the public debate about AI: the doomers are Terrans, and the people who want to press on toward AGI are Cosmists.

Like Hugo, I’m a Cosmist. Should we, in the abstract, decline to develop AI so that humans can stay at the top of the roost forever? To me that’s a bit like the philosophical question of whether we should roll back to the Stone Age, because maybe people were happier then. The story goes: Back then they only worked four hours a day, and the rest of the time they were hanging around, making love, running through the forest with their kids hunting and gathering. Summer camp all year round… at least until you got malaria and died at 35. To me these questions are somewhat imponderable. There is not a clear basis to use to compare two alternatives as different as “Stone Age vs. modern” or “AGI era vs. human-dominated era”. It’s a bit like asking whether I’m happier now or was happier as a little kid. I was pretty happy then and I’m pretty happy now, and they’re just different states of consciousness.

But my own feeling is that evolution has been coursing forward for a very long time, and there’s no more reason to stop here than there was to stop at single-celled organisms, or reptiles, or early mammals, or apes. People can have different views on that, and the Terrans do.

So on Hugo’s map the fight between the doomers and Big Tech is Terrans versus Cosmists, and I’m on the same side of that line as the big labs. But I feel like Hugo’s map of the AI-politics landscape needs updating—because the more consequential division now runs through the middle of the Cosmist camp… and it’s about who gets to do the building: whether the power to develop advanced intelligence ends up concentrated in a few hands or broadly distributed. That division cuts across optimist and pessimist, and across left and right… and we may argue that it decomposes Cosmism into two flavors, which I am giving names here now because they don’t seem to have others:

• Centralized Cosmism has transformative intelligence arriving with its development and ownership held by a small number of organizations.
• Open Cosmism, which is my position, has participation in creating these new minds broadly distributed.

The history of the Terran side is relevant here. The idea that only a small elite group should be allowed to work on AGI has been part of the doomer world for a long time. As I remember the discussions from a couple of decades back, when Nick Bostrom was more of a doomer than he is now, he and Eliezer Yudkowsky were floating ideas along the lines of: some global authority like the UN should enforce a rule that only one specially chosen team gets to work on AGI, locked away in a basement somewhere. Well, it’s not going to be a handful of intellectual superheroes from MIRI and the Future of Humanity Institute in a cell underneath a UN building in Geneva. It’s going to be Dario, Sam and company. But it’s the same basic principle, which means the doomers come pre-primed for exactly the sort of regulatory capture the big tech companies are pushing toward.

So a primary point I want to make here is: The alignment of forces is different from what the public debate suggests. A Terran who wants development stopped is not going to get it stopped. What the Terran campaign is far more likely to get, once it has been through the process of political compromise, is restrictive machinery that reserves development for the few companies able to satisfy it – which is to say, centralized Cosmism, delivered with Terran help. And the two sides can go on sincerely disagreeing with each other the whole time.

My judgment is that capture by the incumbents becomes very likely whenever restrictive permissioning gets built without comparably strong institutions protecting independent expertise, entry by newcomers, and the right to appeal. The rest of this post argues that we’re assembling exactly that combination now.

Not-too-hidden connections

We don’t need leaked messages or other proofs of surreptitious conspiracy to see that the corporate world and the safety-philanthropy world are building AI oversight together. Such evidence might turn up eventually, but we can basically see the dynamic that’s unfolding from purely public indications.

The Frontier Model Forum’s AI Safety Fund, established in 2023, is a collaboration supported by Anthropic, Google, Microsoft and OpenAI together with philanthropic partners including Jaan Tallinn. Among the things it funds is research on standardized third-party evaluations. So the companies whose systems are going to be evaluated are among those paying for the work on how to evaluate them.

Coefficient Giving, one of the big funders of AI safety work, plays a role that goes way beyond approving grant applications. In its staff’s own account, the work includes shaping the plans of organizations, setting up new ones, recruiting founders for projects that Coefficient itself sometimes dreamed up, and convening people around problems that need coordination. By their own description, they are deliberately building an institutional ecosystem.

Fathom’s June 2026 account of independent verification describes proposals in Congress converging with the positions of Anthropic, OpenAI and Google DeepMind. There are disagreements among them, over how much power government should have and how independent the evaluators need to be, and the report records those too. But the overall picture is of an oversight architecture being worked out jointly by a small, connected set of players, which includes the companies to be overseen.

A lot of this runs through the Effective Altruism movement—a name I’ve always found a bit ironic since what comes out of it is so often neither ethical nor good for humanity, though I can believe the sentiment is real for many of the people involved. This is the movement Sam Bankman-Fried came out of, along with a whole bunch of other influential Silicon Valley people. Ask how many of the supposedly independent ethics and safety gurus working in and around Big Tech came up through EA, and you find common historical threads and common funding sources all over the place.

Now, researchers, governments and developers ought to talk to each other, and I have no problem with that. Humans gonna human. My problem is with where all this insider conversation is heading. The institutions that will decide who gets to build advanced AI are being designed and funded largely from inside one well-resourced network of labs, funders and institutes, and the best-resourced members of that network are the incumbent labs. The cooperation is documented. Whether the institutions that come out of it will be independent of the incumbents is the open question, and the next two sections explain why I don’t expect them to be, unless things rapidly change direction.

How sincere opponents end up building a corporate gatekeeper

So why are smart people pushing something as unrealistic as a pause? (I’ll get to just how unrealistic a little later.) Part of it is surely idealism: they really want it, so they push for it whether or not it makes sense in the real world. But some of them can also see what the indirect consequence of pushing is likely to be. You push and push to pause AI. It’s not going to get paused. But you might well get some draconian regulation saying that only a few big companies are allowed to build advanced AI, under the wing of the US government, supplying their tools to the military and surveillance establishments and making money for their shareholders, with no unruly outsiders allowed to play. That would be hard to enforce too, but it’s a far more plausible outcome than a pause.

And listen to what Big Tech itself is saying, which if you boil it down goes something like this. What we’re building is very dangerous. It’s so dangerous it might kill everyone. But we can’t stop building it, because the Chinese are building it. So the best we can do is make sure that only very responsible people like us—people who fully appreciate the danger, since we’re the ones who keep talking about it—have any chance of creating AGI. In other words: let’s make sure nobody can do this except us.

Put those together with a government that mostly doesn’t understand this stuff very well, and look at the incentives. Someone focused on catastrophic risk tends to see a smaller number of independent developers as a safety benefit in itself: fewer places to monitor, fewer actors to coordinate. An incumbent company sees a competitive benefit in expensive certification, officially recognized evaluators, and restrictions that unfamiliar or underfunded competitors will struggle to meet. A government official is glad of an oversight structure that comes supplied with the expertise the government doesn’t have. These three don’t need to agree about what it’s all ultimately for. One of them hopes the machinery will stop development, and another expects it to license development to continue.

Here is how it plays out politically. A demand to stop advanced AI runs into pressure to preserve development that is economically and strategically valuable. The compromise is to permit “responsible” work. Then somebody has to certify what counts as responsible, and certification needs standards, evaluators, access, credentials and resources – all of which the established companies are in a position to help supply. The restrictions tend to arrive first, because the incumbents can live with them. Independent technical capacity, affordable compliance, alternative routes to assurance and enforceable rights of appeal tend to arrive later or never, because nobody with power needs them. So the machinery for saying no gets built, and the protections that were supposed to let outsiders earn a yes stay at the level of promises. From then on the arrangement works for the organizations it was meant to constrain.

The most consequential form of exclusion doesn’t even need to name a competitor. Suppose acceptable safety comes to be defined around centralized hosting, frozen models, and an operator who can revoke every copy—which is to say, around the way the big labs already build and deploy their systems. An AI system with a different architecture, a decentralized one or one that keeps on learning, is then disqualified before anyone has looked at whether its different safeguards work. A standard that looks neutral has picked a technological future for us, and it’s the incumbents’ one.

Plenty of the people involved mean well. Yoshua Bengio I know to be a super nice guy, good-hearted, deep-thinking and smart; Geoff Hinton, whom I don’t know as well, probably is too. People like that are pushing for government regulation for the common good, and I think they just don’t have the political savvy to see that this is leading us straight toward a very counterproductive and destructive sort of regulatory capture. I think plenty of folks in the doomer and Big Tech worlds do see it. And some of the proposals on the table try to guard against it. Fathom explicitly advocates separating the builder, the tester and the standard-setter. MIRI’s stated objective is a worldwide halt that includes the incumbent labs, which is a long way from an Anthropic monopoly. The AI Futures Project’s Plan A explicitly proposes transparency, and letting many companies in different countries catch up. But proposals don’t get enacted whole. That a proposal looks admirable in its completed form says little about the order in which its pieces become politically achievable, and the order is what decides the outcome. Sincerely opposing corporate power doesn’t stop you from helping to build powers that corporations will later dominate.

You can watch this happening right now. In September, Bernie Sanders and Greg Casar introduced a bill that would ban the development of superintelligence outright, with up to 20 years in prison for individuals who break the ban, and would pause all advanced AI development until a new federal Department of AI has written safety rules, after which advanced systems would need federal approval before they could be deployed. Several employees of the leading AI companies endorsed it! Now ask yourself who is going to have the lawyers, the compliance staff and the standing with the new department to get those approvals.

What I mean by “capture” is basically: Governing institutions getting redirected toward private interests at the public’s expense. That’s a different thing from regulation as such, and from concentration as such; deregulation can entrench incumbents just as well. So it tells you very little whether a document uses the word “safety” or the word “innovation.” What you have to ask is who controls the standards, who has access to the evidence, how outsiders are treated, and whether decisions can be challenged.

What capture did to finance and medicine

If you think all this talk about regulatory capture sounds speculative, take a quick look at the two big industries where it has already been followed to the end. Each of them deserves a study of its own; here I’ll just give a capsule summary… accept my apologies for any dyspepsia incurred!

Finance (the most expensive case so far…)

In finance, the major banks and other financial institutions have so much influence over the federal regulators that, while some protection of the consumer does happen, a large part of what financial regulation does is protect the banks themselves.

The classic case is the credit rating agencies. Since the 1970s the US has had officially recognized rating agencies, whose ratings are written into the rules about what banks, insurers and funds could hold. Three firms had nearly all of the business, and they were paid by the issuers of the securities they rated. The Financial Crisis Inquiry Commission, after the 2008 subprime mortgage crash, later reported what this produced: from 2000 to 2007 Moody’s alone gave its triple-A rating to nearly 45,000 mortgage-related securities, and 83 percent of the ones it rated triple-A in 2006 were eventually downgraded. The Commission concluded that the crisis couldn’t have happened without the rating agencies, and that the financial industry had spent three decades getting oversight stripped back in favor of its own self-policing. And the system learned almost nothing. Years after the crash the same three agencies still had more than 96 percent of outstanding credit ratings, the sector was more concentrated than ever in a few very large institutions, and the formation of new banks had fallen from about a hundred a year to about three, with the weight of new regulation among the reasons.

It’s good to be a bank! The government gives you a uniquely sanctioned right to buy low and sell high—to take in money cheaply and lend it out dear—and among the things you can buy, once you’ve made enough money doing that over and over, are the regulators who make sure nobody else gets in on the game.

Now look at how that system treats an alternative that came from outside it. I know better than most that the crypto world has its own problems. There are a lot of scams and a lot of nonsense. But take stablecoins. Why shouldn’t a stablecoin that is audited and regulated by the government, so we can be sure there’s no scam going on, be allowed to pay interest? A dollar stablecoin is backed by Treasury bills and similar assets, which pay interest. Yet the GENIUS Act of 2025 forbids the issuer to pay any interest to the people holding the coins, and the bank lobby then set out to stop exchanges from paying it either. That fight was the central dispute that kept the Clarity Act, the broader bill setting rules for crypto markets, stalled in the Senate for nearly a year after the House had passed it 294 to 134. The compromise reached in May 2026 bans interest on stablecoins that are simply held, allowing only rewards tied to activity such as payments, and in September the bill failed a procedural vote in the Senate anyway, 49 to 50, with the banks’ objections to stablecoin rewards still among the reasons.

The banks’ argument is that people would move their deposits, and the banks would have less to lend. Which is to say, the only real reason is that the banks don’t want that threat to their business model. A competitor would be offering ordinary savers a better deal, and the banks’ remedy is to get the better deal made illegal instead of paying their own depositors more. And the law doesn’t get fixed, because these big companies in essence own the regulators.

Or look at startup investment. Why do you have to be an “accredited investor”—for an individual, that means an income over $200,000 or a net worth over $1 million, not counting your home—to put money into most early-stage companies? At some point this may have been needed to protect people against scams. But now we have the internet, with endless ways to inform people about what’s legitimate and what isn’t, and people are getting scammed in all sorts of far worse ways than by investing in early-stage tech startups. We let people play the lottery, where the odds are calculated against them in essentially every case. We let them go to casinos and gamble their money away. But we don’t let them put some of their hard-earned money into startups, and pull it out and put it into other startups, based on their own intuition. Where’s the fairness in that? Well, there’s a financial establishment that makes its money from the traditional methods of equity investment.

ICOs and related crypto mechanisms can let ordinary people invest bits and pieces of money, in a liquid way, in early-stage ventures. I came to understand this scene pretty well when launched SingularityNET in 2017. Plenty of the ICOs offered in this era were scams, and clearly rules were needed to minimize these, but there’s no reason this can’t be regulated in a sensible way, with the people committing fraud prosecuted and ordinary people allowed into the non-scam detals. Alongside SingularityNET and a bunch of scams, there were also quite a lot of other legitimate and innovative projects raising money via ICO that would never have gotten funded in the traditional venture ecosystem, due to biases regarding business model, geographical location, team composition, and so forth.

But what we got instead of sensible regulation to different the real projects from the scams was, basically, haphazard enforcement of very poorly articulated policies. By the end of 2017 the SEC’s chairman was saying that the ICOs he had seen were by and large securities offerings, which in practice meant full registration that hardly any startup could afford, or selling only to accredited investors, or staying out of the US. Meanwhile ICOs proceeded apace.

Wall Street and crypto are getting closer to each other now, and the establishment is starting to make money in crypto too. But on the whole crypto, alongside its less savory aspects, has a democratizing side, and the regulators have had a lot of trouble with that.

The effect of keeping crypto marginalized is to push wealth inequality even higher, and also to squelch innovation. Precisely because crypto has been kept out of the mainstream, it has stayed a crypto-bro, crypto-cowboy sort of thing, and a lot of people in the mainstream economy have wanted nothing to do with it, including ordinary investors and including people with cool ideas for tech companies. If crypto regulation were clear and reasonable, a lot more people with wild ideas that don’t pass muster with the small elite of Silicon Valley super-angels and VCs could get funded by ordinary people instead. That mostly isn’t happening, because for most people crypto is still a legal gray area (though it’s pretty legit compared with five years ago) and still just hard to do.

Why can’t you open your ordinary banking app and move money straight into a Bitcoin wallet, and from there into whatever you like? As far as I can see, mainly because your bank would rather hold on to your money and earn interest on it, and the law sanctions its unique right to make money from your money. That’s regulatory capture in banking.

Medicine (the most sickening case…)

Of course we need medical regulation, for everyone’s safety. Ordinary people who don’t know medical science need protecting from the kind of quack medicine we had 150 years ago—literal snake oil, sold for all sorts of purposes it was never going to cure, with no way for the buyer to tell the difference between this sort of nonsense and genuine therapies.

But look at what the system we built does in practice.

In 1995 the FDA approved OxyContin with a broad indication on the strength of a single adequate trial, lasting two weeks, in osteoarthritis patients. The two principal FDA reviewers on the application both later took jobs with Purdue, the drug’s maker. In 2002, when the agency asked ten outside experts whether the broad indication on opioid labels should be narrowed, eight of the ten had financial ties to drug companies, Purdue among them, and they advised against. The trial design the FDA went on to accept as evidence that opioids work for chronic pain came out of private meetings between FDA officials and drug company executives, which companies paid up to $35,000 apiece to attend.

Opioid overdoses, prescription and illegal together, killed about 806,000 Americans between 1999 and 2023, in an epidemic whose first wave the CDC traces to the rise in opioid prescribing in the 1990s. Purdue pleaded guilty to three federal felonies in 2020, and as of that year the FDA had still not done a root-cause analysis of its own part in the disaster.

And OxyContin is one item in a whole litany of unsafe medications that got approved through this elaborate and enormously expensive safety process. The process means it takes something like a billion dollars, and often a decade or more, to get a therapy from invention to release. Between 1950 and 2010 the number of new drugs approved per billion dollars of R&D fell by half about every nine years, roughly eighty-fold in all, with the steadily rising bar set by cautious regulators among the causes.

So you have to be super rich to push anything through, and then the rich companies push through things that kill people anyway. That’s not just because science is hard. It’s because the medical-governmental establishment is somewhat corrupt, even in the US, which is neither the most corrupt nation on earth nor the least. About two-thirds of the FDA’s budget for regulating human drugs comes, through user fees, from the companies whose products it reviews.

Of course you want monitoring for safety, but there are loads of publications explaining cheaper, faster, more sensible ways to do it. A big part of the reason the pipeline stays so long and so expensive, in my view, is that the pharma companies don’t want anyone else to be able to bust in. If your startup invents some amazing new medicine, it’s very hard to bring it to patients any other way than by selling the startup to a pharma company with the bank account and the patience to carry it through the whole long process. Incumbents get the same kind of protection on the delivery side. In most US states you can’t open or expand many kinds of health care facility without a state “certificate of need,” and in 29 of those states an existing competitor can formally object to your application, an arrangement the Justice Department and the FTC have criticized for years as a way for incumbents to keep newcomers out.

Then look at alternative medicine: traditional Chinese medicine, and the various other kinds of medicine outside the Western mainstream. Some of it is quackery, and some of it is dangerous. Some of it also does a great deal of good, and a system that wanted to learn would be sorting the one from the other as fast as it could. The American medical establishment has more often tried to shut the alternatives down. In 1987 a federal court found that the American Medical Association had run an unlawful conspiracy “to contain and eliminate” the chiropractic profession, in a campaign going back to the early 1960s. The judge accepted that the AMA’s main motive was a sincere belief that it was protecting patients, and found the conspiracy illegal all the same.

It doesn’t have to work that way. When I lived in Hong Kong, you could go to pretty much any random doctor, and for acute problems, where Western medicine is usually better, they’d prescribe Western medicine. For chronic problems, where Western medicine often doesn’t have much to offer— migraines, irritable bowel syndrome, various long-standing infections, chronic fatigue syndrome—they would tend to turn to traditional Chinese medicine, which in my own impression, and that of a great many other people in Asia and here, works better for that sort of thing.

I wish there were a drug to cure chronic fatigue. I wish there were a drug to cure irritable bowel syndrome, which I’ve had on and off through my life. There isn’t, and herbal and holistic Chinese medicine tends to do better. The system in Hong Kong is built for this: Chinese medicine practitioners are registered by statute, the public Hospital Authority offers integrated Chinese-Western services in 26 of its hospitals, and a dedicated Chinese medicine hospital opened in December 2025.

And the payoff from taking traditional remedies seriously can be very large. Artemisinin, now used wherever malaria is found and credited with saving more than 100,000 lives a year in Africa alone, came out of a Chinese screening of traditional herbal remedies, with clues from the ancient literature on how to extract it.

So why isn’t this a bigger thing in the US? Well, herbs are open for anyone to buy, and lifestyle changes are open for anyone to make, so the pharma companies can’t make a pile of money from them. A remedy that’s been in use for centuries is hard to patent, so no company can earn a guaranteed profit from it without competition, and the trials that would tell us what works don’t get run.

The FDA has had a dedicated pathway for botanical drugs for about twenty years. By 2018 it had received more than 800 applications and pre-application inquiries, and to date it has approved four products. A former FDA official who worked on that pathway named patents as one of the main reasons for the low numbers. The NIH center charged with studying complementary and integrative health gets about $170 million a year, roughly a third of one percent of the NIH budget. What you end up with is very tight regulation of anything sold as a medicine, and very little effort going into pushing through the medical system the things that won’t make a pharma company a pile of money.

We have better tools now than gatekeeping by incumbents. With the modern internet we could set up a decent reputation system that lets people who know medical science rate claimed therapies in a decentralized way, so that people could judge for themselves: what’s the evidence for this or that therapy, and what’s the background of the person endorsing it? In the same way you could have decentralized rating agencies for investment opportunities, where experts in quantum computing or nanoscience or international trade rate ventures in their own areas, which would make startup investment a lot safer for ordinary people than it is today. The government would do much better to put its money into fostering decentralized rating systems than into feeding more and more of it to the big banks and big pharma companies and their shareholders.

First Finance and Medicine, Next AI?

Put the two cases side by side. In both, the evaluators are recognized by the regulator and paid by the evaluated; the standards are shaped by the incumbents and built around what they already do; the price of admission is set at a level only the incumbents can comfortably pay; and the alternatives that come from outside, crypto in finance and traditional therapies in medicine, are kept at the margins while the mainstream press rarely makes their case. The institutions lose the ability to learn from their failures, because the people who failed are still the ones in charge. The public doesn’t even get the safety that was the reason for the whole arrangement, since both catastrophes came out of approved products. And creative exploration that would very probably have been for the common good—working out how crypto can best serve ordinary people, working out which traditional therapies can improve everyone’s health—has been held back for decades.

This is all pretty well-known stuff. And so far the tech industry hasn’t really been sucked into it. But the same features are now taking shape for AI, proposed in good conscience: officially recognized evaluators, with the labs helping to pay for the evaluation science; safety standards liable to be built around the labs’ own way of building and deploying systems; compliance costs that a frontier lab can absorb and an open-source project or a university group can’t. Capture of AGI, with the media’s help, would stifle exploration for the good of everyone in just the same way. The difference is in the stakes. In finance and medicine what got stifled was innovation in business and economic fairness, and in human health and longevity, which is bad enough. With AGI it would be innovation in a domain more consequential than either, where the outcome may be the difference between dystopia and abundance. And finance and medicine at least had a long history of open experiment behind them before the gates went up. With AGI the gates would go up while most of the possible kinds of minds are still untried.

The press is free to those who own one—and fairly tractable to rent if you’re a massive corporation…

The media is captured in both of these cases too.

Financial institutions—investment banks, commercial banks, and private equity firms—don’t exactly tell media what stories to run on a day by day basis (or at least not usually) but they own significant shares of media companies, they spend big on ads and lobbying, and they provide debt financing that media companies often require. The influence all this has on media content is real, unsurprising and well documented. The odds of a newspaper reporting a scandal within a company go way down if that company sponsors them heavily with ads. The language used to describe a corporate merger gets much more rosy if the financial journalist went to the same business school as the CEO involved. Profits on the part of the media company’s lender are more likely to be reported than losses—etc.

Meanwhile, look at the dismal nature of crypto media, which is mostly “buy this token, buy this token, buy this token,” which puts crypto in a bad light for a lot of people. The mainstream media has never really picked up the narrative that crypto could be used to democratize and decentralize things. You could say they’re just not very imaginative, so they take the easy route and go with the obvious stories about CZ and Sam Bankman-Fried and whatever fraud got prosecuted lately, and that’s kind of true. On the other hand, the financial dependency of media companies on traditional banks has got to help push them in this direction.

On the health side, many normal people have heard of acupuncture and herbal medicine, and there’s niche media that takes them seriously. But the mainstream media very rarely makes the case for alternative medicine as a meaningful paradigm, and it’s probably not coincidental that there’s lot of pharma money in the picture. In the first five months of 2025, drug ads took up almost a quarter of the advertising time on the evening news programs of ABC, CBS, CNN, Fox News, MSNBC and NBC, and the US is one of only two countries that allow such advertising at all.

I haven’t done the full analysis of exactly how far the big banks and big pharma have bought the media, but I doubt it would be hard…

Basically—when it comes to the programming of the average person, the media follows the dollars, right along with whatever the large corporations that have in effect bought the regulators are pushing for.

The media help decide which futures look possible

In AI, as in finance and medicine, part of the struggle is over what counts as a reasonable question to ask in the first place. And media is very critical here. What the media presents right now is a ping-pong match. Will Big Tech kill us all, or will Big Tech save us all? Back and forth it goes, until people think that’s the only game in town. The message that there could be another way than Big Tech altogether doesn’t get presented at all. The alternative ways of making AI ethical, and for that matter the alternative ways of making AI in the first place, get essentially no coverage.

How does that happen? The EA side buys some media coverage, and Big Tech buys a lot more.

Some of it is plain commerce. In April 2026 OpenAI announced that it had acquired the technology talk show TBPN and placed it inside OpenAI’s strategy organization, with a promise of editorial independence. OpenAI’s earlier agreement with News Corp ties its products to content from major publishers. Neither arrangement proves that any particular story was interfered with, and it doesn’t need to. An AI company that owns a media outlet and has commercial deals with major publishers has a seat inside the business of covering AI, whatever promises of editorial independence come attached.

On the doomer side, MIRI has published a communications strategy that explicitly targets policymakers, their advisers and the public, via interviews, podcasts, opinion pieces and a sustained flow of explanatory material. That’s an openly declared campaign to shape the political environment.

Then there’s the funding of journalism itself. The Tarbell Center for AI Journalism lists Coefficient Giving, the Survival and Flourishing Fund and the Future of Life Institute among its supporters, and in October 2026 it announced a program that aims to distribute at least $10 million to newsrooms and media ventures by the end of 2027. Tarbell says that neither it nor its donors control the content, conclusions or framing of the reporting it supports, and that donors don’t select the fellows or the stories. I have no evidence to the contrary, and I’m not suggesting anyone gets marching orders. Nobody needs to give any. When a program that trains and funds AI journalists is supported by funders prominent in the catastrophic-risk world, the sense of which questions are worth asking gets shaped well before any story is assigned.

And the neat picture of corporations on one side and doomers on the other falls apart altogether when a corporation funds advocacy for stronger oversight. In July 2026 Anthropic disclosed that its total support for Public First Action’s public-education and policy work had reached $40 million, and said the money could not be used to influence candidate elections. That’s policy advocacy rather than newsroom funding, but it shows how little financial separation there is between the two supposed sides.

So the issue goes deeper than whether any given article has been paid for. It’s whether the overall allocation of money, access and recognized expertise leaves the public with a diverse enough set of explanations to choose among, and what evidence there is suggests it doesn’t. A Reuters Institute study of eight months of coverage in six UK outlets found a debate led by industry, in which AI was frequently presented as a private commercial concern. A case study of New York Times reporting from January through March 2024 found that 67 percent of the sources quoted worked in commercial technology, with experts from outside industry often cast as skeptics or outsiders. These studies cover particular outlets and periods, and more are needed, but they fit my own experience.

There’s a big difference between being invited to criticize the recognized builders and being recognized as someone who understands a different way to build intelligence. Loud disagreement doesn’t widen the underlying assumptions, either. “Let the labs advance” and “make the labs slow down” both leave the same labs, the same architectures and the same ownership arrangements at the center. When a set of interconnected corporate, philanthropic and media institutions keeps offering the same narrow menu, audiences mistake agreement inside one ecosystem for independent confirmation. Shared affiliations don’t make anyone’s conclusions wrong; they do mean a consensus can look broad and still come from one place.

I see this with my own arguments all the time. I say that open, participatory research may help us discover architectures with stronger moral capacities, and what a lot of people hear is “an entrepreneur thinks innovation is more important than safety”—which is a different argument from the one I made. Continual learning gets presented as something that undermines safety guarantees, with no mention that it may also be an ingredient of moral development. Decentralization gets presented as loss of control, with no mention that it may also be a way to preserve independent inquiry and participation.

Not every journalist works this way, and AI coverage as a whole is hardly dominated by doomers: the Reuters Institute’s broader review of the research finds it to be often positive and led by industry. That fits what I’m saying perfectly well, though. Whether the tone is boosterish or fearful, the same institutions supply the experts; the experts’ questions organize the coverage; the coverage informs policy priorities; policy and funding then favor the institutions everyone already knows; and their resulting dominance makes their expertise look even more uniquely authoritative than before. Most people never hear the alternative stated clearly enough to accept or reject it. Counting how often “open source” gets mentioned won’t reveal this. You have to look at whether the developmental argument about safety makes it into the reporting at all.

But is there also an actual conspiracy?

When I first started thinking hard about what’s going on between the doomers and Big Tech, it felt to me like it had to be some sort of conspiracy.

Now, I’m not much of a conspiracy theorist. I don’t buy the Kennedy assassination theories. I think we did land on the moon. By and large I’ve found in life that when you have two explanations on offer, one of them rampant human stupidity and the other some very subtle and complex set of machinations that nobody can quite spell out, it’s usually just human stupidity.

Take 9/11, which I’ve thought about a fair bit. My first AI company, Webmind, had its office at 50 Broad Street, and I used to walk through the World Trade Center every day on my way to work; somehow I moved away from New York about two weeks before the towers came down. There were things about that event that were never fully disclosed. But if somebody had schemed the whole thing, it would have come out by now. I think we just had sloppy anti-terrorism, and terrorists who were clever and determined.

On the other hand, I have no doubt at all that a great many secret conversations go on in back rooms between this powerful person and that one.

Anyway, for a while I had myself convinced there was a big conspiracy here, and you can follow the threads in a conspiracy-theorist sort of way. You see the funding connections. You see the common organizational and ideological affiliations, a lot of them running through EA. Take Jacob Coxon, the researcher who quit Anthropic in September with a public warning that AI could kill us all, and overnight became the face of AI safety. As I wrote at the time, he didn’t come from nowhere. Before he ever worked at a frontier lab he was part of the London rationalist scene, living at Newspeak House, a residential college for young people interested in technology and politics. When it was time to announce his exit he turned for help to a friend at an AI safety advocacy nonprofit, and within minutes his posts were being shared by that friend and other AI safety advocates. Within days the heads of the rival labs had come together to agree that it was time to slow down. Coxon says he decided to leave on his own, and I believe his warning was heartfelt.

Sometimes I think the public dialogue runs something like:

“We don’t know if our AI is going to kill us all…”

“Maybe you should stop building it then?”

“No – then China will take over the world. You’d rather be killed by our AI than by a Chinese AI if it comes to that, right?”

“Well… maybe… I guess… but…”“Look, just shut up and give us two trillion dollars for our IPO, and at least we’ll make sure we’re the only ones who kill everyone. No, no, I mean: give us the money because we’re the only ones smart enough to have a prayer of not killing everyone.”

Anyway it’s easy to start speculating—what signals passed between these people involved in this dialogue? What conversations were had in this or that guy’s hot tub in Palo Alto?

I still suspect there’s some element of at least quasi-conspiratorial activity: a fairly small group of people, spread across corporate policy teams, funders, advocacy organizations and political offices, who have privately agreed on where they want AI regulation to end up, and who present their public interventions as more independent of one another than they are. Most of the researchers and journalists taking part in the wider debate wouldn’t need to know anything about it. A group like that can coordinate warnings and endorsements so they look like a spontaneous consensus. It can trade its support for restrictions for influence over appointments, standards or exemptions. It can make funding or privileged access conditional on promoting one particular model of oversight, or on leaving competitors out. I can’t tell you who is in such a group, and I make no accusation against any organization or person named in this post. But the regulated shaping their own regulation, partly out of public view, is part of the record in both finance and medicine, and I’d be surprised if AI turned out to be the exception.

Proving existence of an actual conspiracy between big tech and doomers would be a different thing. It would take authenticated communications showing a common direction that was being concealed; funding agreements with undisclosed policy conditions; drafting histories that tie an exclusionary aim to specific provisions; corroborated records of promised appointments or exemptions; or evidence of hidden editorial instructions. A grant, a friendship, a shared belief or a publication embargo proves nothing. No evidence of that kind is public today, from what I could find. But that doesn’t mean it won’t turn up eventually!

The inside story of the rating agencies was laid out by the Financial Crisis Inquiry Commission more than two years after the crash, and with opioids it took about two decades of litigation and reporting. The AMA’s campaign against chiropractic began in the early 1960s and was ruled an unlawful conspiracy in 1987. By the time the proof of this sort of thign arrives, the arrangements it concerns are already in place.

But anyway, the more I thought about the conspiracy question, the more I realized it mostly doesn’t matter. Say there was never a meeting in a hot tub in some mansion in Palo Alto or Mountain View where a bunch of doomers and Big Tech execs decided to go forth and play the media. Then what we have is an emergent dynamical phenomenon. The doomers put their pie-in-the-sky, unrealistic stuff about pausing and unplugging AI out there. The Big Tech guys do their corporate thing of ethics-washing, playing the mass media and the mass mind with all this talk about how dangerous their stuff is, and therefore how valuable their stuff is, in the run-up to their IPOs. The media buys the idea that the ping-pong between doom and Big-Tech-as-savior is the only game in town. And the government, most of which doesn’t understand any of this very well, drifts toward regulatory capture. Conspiracy or emergent dynamics, in the end it comes to the same thing. We should investigate hidden intentions where we can, and oppose the outcome whatever the investigation finds.

Capture can select for the wrong kinds of minds

The familiar objections to capture are about wealth, accountability and freedom, and finance and medicine show how well founded they are. With AGI there’s a further one: capture is likely to make the intelligence we end up building less safe.

Sure, restrictive rules in the US protect against some dangers. They protect against rogue AI startups or crypto collectives somehow creating a nasty AGI, even nastier than what Dario and Sam are building, outside the eyes of the regulators. Of course they don’t protect against Putin doing the same thing, or people in Iran, or a bunch of smart people in Moldova or Azerbaijan or Turkmenistan. There’s a conceit in the US that only recent Stanford grads in Silicon Valley are smart enough to build AGI, which is very far from true. AI was conceived largely in the US, with a lot of help from other countries and mostly on government research funding, but the cat is long since out of the bag, and there are people doing AGI R&D all over the world.

And while you’re restricting some dangers, you’re also potentially squelching the development of fundamentally safer ways of building AGI. Because what is Big Tech building now? AI systems that have no moral agency, because they have no real agency at all. These systems don’t know who and what they are. They’re models trained in a batch on all the data on the internet. People try to stick a constitution into the training data—“be good”—and think that will make the thing be good. No, Dario, that doesn’t work. And since putting the constitution into the training data, in various fancy forms, doesn’t work, and your AI will do random bad stuff anyway, you put guardrails on the outside to stop it from doing the random bad stuff. Obviously that doesn’t work either.

Think about raising a kid to be ethical. Suppose your method is to give them a stack of books on ethics to read. Well, they’ll treat those as a few among the many books they read. So then, when they start doing bad things, you zap them every time they do something you don’t like. What could possibly go wrong? Surely your kid will never use everything they learned from all those other books to figure out how to get around your rigid and inadequate guardrails. Come on. You know that doesn’t work.

What could work is to make an AI agent that knows who and what it is, and knows who and what you are. One that learns from every one of its interactions as it goes through the world, continually, instead of learning in batch mode and then just doing inference on what it learned until six months go by and you train a new model. One that builds a model of itself and a model of you. Then you take that system, with a real self-model, a real world-model and some agency in choosing its actions, and you carry out beneficial actions together with it in the world, the way you do with your own human kids. You don’t raise kids mainly by handing them a book of ethics and zapping them when they misbehave. They’re agents. You know who they are, they know who they are, they’re modeling the world, you’re part of their world-model and they’re part of yours. You model good behavior for them and you do good things together with them, so they get into a shared mind-space with you of doing beneficial things, and that shapes their minds and makes them moral agents in their own right. It doesn’t give you an absolute guarantee, with kids or with AIs, but it biases things in that direction. This is what we’re starting to do now with the Omega agents, in the Omega projects at SingularityNET and BGI Labs, built on the Hyperon AGI toolkit.

There’s a more theoretical way of saying the same thing. In his 2008 paper “The Basic AI Drives,” Steve Omohundro examined the pressures pushing an AI system toward self-improvement, resource acquisition, self-protection and the preservation of its goals. It’s an important warning. What it doesn’t license is treating all possible minds as though their architectures and motivations were interchangeable, which is how the argument often gets used. My recent Basic AI Drives blog post series develops the alternative. Take a system with multiple objectives, limits on the means it will permit itself, care for other beings and for their choices, values woven through its cognition, self-modification that gets checked, and companions that really are independent of it. A system like that may develop a different set of self-reinforcing tendencies – what I’ve been calling beneficial basic AI drives – which include preserving meaningful oversight, repairing its own values, inventing ways around moral conflicts, and empowering other beings.

Let me suickly sum up for those who haven’t read the series. An attractor basin is a region of a developmental landscape such that systems starting inside it tend toward a characteristic pattern, and tend to return to that pattern after disturbances that aren’t too large. A beneficial basin doesn’t make everything come out nicely all by itself; it has to be supported by cognitive and social mechanisms that we build, test and maintain. And Part 4 of the series makes the argument that bears most directly on capture: asking what fraction of minds ends up in a given basin is an incomplete question unless you also say which minds get built. How much rain reaches an ocean depends partly on where the rain falls, and our development institutions help decide where it rains.

So an oversight regime can reduce the failure rate of every architecture currently available and still increase the long-term danger, by suppressing the discovery of better ones. That’s the AGI version of what the approval system has done to drug development, with far more at stake. By trying, probably unsuccessfully, to slam down on the risk of bad behavior by these early proto-AGI systems with no moral agency, you’re potentially stomping out the research on building AGIs that have it. And Big Tech isn’t doing much of that research. It’s not exactly that they don’t care. Partly they don’t know how, because they’ve mostly hired people with a fairly narrow sort of AI background who are good at building scalable transformers. And partly they’re in a race to build better and better transformers than each other and than the Chinese, which leaves no time to waste on funky alternative models of AGI with moral agency.

It’s true that openness can turn up dangerous designs as well as beneficial ones, and a fair comparison of trajectories has to count both effects. What I object to is counting only the one that supports the policy you already prefer. The safety question has to change. Besides asking how to put guardrails on the most commercially successful architecture, we should be asking which research environments make it more likely that we discover architectures with intrinsically stronger moral capacities.

By moral agency I mean more than obeying a specification. I mean understanding morally relevant situations, recognizing interests that have been overlooked, caring about what others choose, and arriving at better judgments through experience. A morally capable system might identify a harm its designers failed to anticipate, where a merely compliant one would just avoid the prohibited phrases. The target is judgment integrated with a durable concern for others, which is a good deal more than autonomy alone. And the architecture that is most profitable to scale isn’t necessarily the one best suited to developing it. If it lacks essential machinery for moral learning—the ability to keep learning from its own experience, for a start—then making its behavior more controllable doesn’t supply that machinery, and locking research into that route means perfecting the supervision of the wrong developmental starting point.

The alternative has to be explored before it can be proven

I favor open, decentralized AGI development in large part because I believe it can sustain a much deeper diversity of architectures, and bring many more of the world’s people into the development and training of these systems, than a handful of corporate labs ever will.

By participation I mean a lot more than giving different communities a chance to rank a chatbot’s answers. They should have a hand in shaping the problems an intelligence recognizes, the relationships through which it learns, and its understanding of dignity, obligation, flourishing and choice. Different researchers, likewise, should have room to investigate different kinds of minds, as opposed to cheaper versions of the one commercially dominant design. My colleagues and I are pursuing some of the possibilities through Hyperon and Omega, including ways of building AI that don’t need a hyperscaler’s server farm and can run across a network of small computers, on decentralized platforms like the one we’ve built at SingularityNET and will be rolling out in better form with ASI:Chain. And we should all want other groups out there pursuing approaches we haven’t imagined.

Openness is no guarantee of wisdom. Open networks can produce dangerous systems; in my story “The Nine Days of Madness,” it’s a reckless extension of an open experiment that helps bring on the catastrophe. An open ecosystem can also collapse into a monoculture, and a public or private laboratory can be home to real diversity. What counts is whether there’s effective participation and freedom of inquiry, whatever the label says. And doing research in the open doesn’t mean releasing every capability without restriction.

So here’s the claim. Decentralized, participatory exploration of architectures is a serious pathway to safety, and it deserves resources, realistic experiments, and institutions that help promising designs survive. Discovering a good design isn’t enough if the surrounding economy rewards manipulability and speed and selects against moral restraint.

What I won’t accept is the demand that this pathway prove itself the best one before it’s allowed to develop. That demand is circular, because the evidence can only come from research that the permissioning regime would prevent. And nobody is asking the incumbent architectures to meet any such standard before they get scaled up. Do we want to clamp down on research into how to make beneficial AGIs because we’re so worried about stopping anyone else from making AIs without moral agency better than Big Tech does?

The pause we would get in practice

As a practical matter, “pause AI” is just not possible. How the hell do we pause AI? To begin with, Trump, Xi Jinping and Putin don’t want to pause AI, and if one of them said he was going to, the others wouldn’t trust him or believe him.

And what would it even mean? You can’t pause AGI or ASI research without pausing all AI research. How do you stop someone from building a trading system? Or an AI for biomedical discovery? Or an AI that proves math theorems? (Oh no, say the famous high-prestige mathematicians, AI shouldn’t prove theorems, because then our Fields Medals won’t be worth as much status! Well…) You can’t stop AGI research without stopping people from making computers smarter at every practical thing, because if you tried, everybody would just stop calling their work AGI. “No, I’m not making AGI. I’m making a self-driving car. I’m making a mathematician’s assistant. I’m making a house-cleaning robot. I’m making an AI to cure cancer and do surgery that people can’t do.” And to do those things the AI obviously needs to get smarter and smarter.

Set aside, too, the fact that anyone with a brain, a computer chip and an internet connection can do AI research. How are you going to stop them from developing innovative new ways of building AI that run across a network of ordinary machines instead of in one giant server farm? You only stop that with some sort of global 1984-plus fascist regime. And if you think through the likely consequences of that—the revolutions and the world wars that would follow—there’s a lot more very clear existential risk there than in anything we face now from AI systems.

The advocates of a pause do have serious proposals, MIRI’s proposed international agreement among them. But designing a treaty is one thing, and having the political capacity to secure it and enforce it worldwide is another, and I see no sign of that capacity.

So look at the fallback, because the fallback is what the campaign for a pause produces in the world we live in: selective restriction. Development that is politically protected, or foreign, or simply less transparent carries on, while visible independent research gets harder to do. The restrictions change who builds advanced intelligence much more than whether it gets built.

In that world, closing off alternative architectures is a safety cost, on top of the economic one. It makes no sense to count the speculative harms of openness while tacitly assigning zero to the speculative benefits of discovering more morally capable forms of intelligence. Both belong in the calculation, along with the risks of transition and of misuse.

Keeping the future open enough to improve it

None of this is written yet. There’s a strong tendency toward regulatory capture, but we don’t have the capture yet.

Amusingly, Trump doesn’t seem to be entirely buying the push for it, as far as I can tell. When the heads of the big labs came out together for a slowdown in September, his response was that a “strong and smart” president is the only guardrail the country needs. Credit where credit is due: he’s leaving things relatively open. Nobody is enforcing a fascist hegemony over AGI development. No ICE-style agents are coming to my house to seize my laptop and make me delete my AI from my servers. Bernie Sanders, on the other hand, has just proposed up to 20 years in prison for developing superintelligence. If we get to the next presidential election without AGI, and we get a more conventional sort of president, they’re more likely to buy the whole regulatory-capture narrative, unless they’re unusually open-minded, which isn’t usually what we get in top-level politicians.

So we have a window. A window of time to develop AGI with real moral agency, using different algorithms from the ones Big Tech is using, which are more capable of promoting the benefit of the human species. And a window to use the glory of the open internet, which is still there, to get past the media ping-pong about whether Big Tech will kill us all or save us all, and wake people up to the idea that there are paths to beneficial AGI and a beneficial Singularity that bypass Big Tech altogether. We’ve got to grab that opportunity. That’s what I’m trying to do with Hyperon, Omega, SingularityNET and BGI Labs, which you’ve already heard plenty about if you’ve been following me for a while.

On the policy side, we should demand oversight that constrains the powerful developers without making them the arbiters of everyone else’s legitimacy. That means technical capacity that’s independent of the labs, conflicts of interest out in the open, affordable routes by which any developer can demonstrate safety, and a credible process of appeal. And these protections have to arrive together with the restrictions or ahead of them. Restrictions now and safeguards for outsiders later is exactly the sequence that produces capture.

We should ask journalists to look beyond who’s funding a given frightening story, and to ask which explanations never get to be stories at all. Alternative research programs ought to be investigated and criticized like anything else, without being automatically filed under opposition to safety. And we should fund the difficult work of comparing developmental trajectories: architecture, motivation, continual moral learning, participation, and the institutional pressures that determine which designs get to mature.

None of this depends on showing that the people involved are cynical or dishonest. Many of them are sincerely trying to save the world. So were many of the people who built the gatekeeping systems of finance and medicine. Sincerity doesn’t settle what an institution does once it exists. Whether some of the participants are conspiring, as I suspect, or merely converging, the result would be the same: the fear of dangerous intelligence used as the justification for closing off routes to beneficial intelligence that have barely been explored. Who gets to hold the keys is only part of what’s at stake. The larger part, I think, is what kinds of minds the institutions holding those keys will allow humanity to discover.

I did a podcast recently with my good friend Joscha Bach, in which, after a bit of Nietzsche (he is German, after all), he moved on to Leibniz and started arguing that this is in fact the best of all possible worlds, and that if it doesn’t seem that way, it’s because we have a limited view. I’ve thought about that in a few ways. One way to look at it is that what our species seems to like best is to get right up to the precipice of destruction and then save the day just in the nick of time. That’s the end of a good Hollywood movie. That’s the hero’s journey, the way you get the maximum personal growth. In that sense, maybe this really is how it’s supposed to go. Maybe defeating regulatory capture of AGI is what we need to do to satisfy our taste for almost wrecking ourselves and then saving ourselves at the last moment. If so, this is the time to do it.

References and further reading

1. Hugo de Garis, The Artilect War: Cosmists vs. Terrans: A Bitter Controversy Concerning Whether Humanity Should Build Godlike Massively Intelligent Machines (ETC Publications, 2005). The source of “artilect,” “Cosmist” and “Terran” as used here.
2. Frontier Model Forum, “AI Safety Fund.” The fund’s own account identifies its corporate and philanthropic partners and its support for standardized third-party evaluations. It documents cooperation; it does not document any agreement to capture regulation.
3. Coefficient Giving, “What surprised our staff about working at Coefficient Giving” (August 6, 2026). See Darius Meissner’s account of shaping plans, establishing organizations, recruiting founders, and convening participants.
4. Fathom, “Independent Verification Goes Mainstream” (June 16, 2026). Describes convergence around verification while distinguishing the proposals’ government powers, financial-independence protections, and approaches to standard-setting. Fathom advocates separating builders, testers, and standard-setters.
5. MIRI, “MIRI 2024 Communications Strategy” (May 29, 2024). Documents deliberate public communications and the objective of stopping frontier development worldwide. The latter is significant counterevidence against reading MIRI’s ultimate objective as an incumbent corporate monopoly.
6. AI Futures Project, “AI 2040: Plan A” (2026). A recommended scenario involving verified restraint, transparency, and dispersing power across companies and countries. It is a proposal and makes no prediction that its institutions will be implemented.
7. Office of Senator Bernie Sanders, “Sanders, Casar Introduce Legislation to Create New Federal Agency to Ban Artificial Superintelligence, Pause Advanced AI Development” (September 23, 2026). The bill would permanently ban the development of artificial superintelligence, pause advanced AI development until safety rules are in place, and subject individuals to up to 20 years in prison. On the federal-approval requirement and the endorsements from AI company employees, see the Associated Press report carried by PBS, “Sen. Bernie Sanders unveils bill to ban artificial superintelligence and create Department of AI.”
8. Financial Crisis Inquiry Commission, “Conclusions of the Financial Crisis Inquiry Commission,” in The Financial Crisis Inquiry Report (January 2011). Source for the Moody’s figures and for the Commission’s conclusions on the rating agencies, on three decades of deregulation and industry self-policing, and on concentration after the crisis.
9. The Bond Buyer, “SEC Staff Has Some Concerns with Credit Rating Agencies.” Reports an SEC staff examination finding that S&P, Moody’s and Fitch together accounted for more than 96 percent of outstanding credit ratings.
10. Roisin McCord, Edward Simpson Prescott and Tim Sablik, “Explaining the Decline in the Number of Banks since the Great Recession,” Federal Reserve Bank of Richmond Economic Brief (March 2015). The authors cite regulation alongside low interest rates and a weak recovery; other Federal Reserve research attributes about three-quarters of the fall in new charters to the post-crisis economic environment.
11. Unchained, “Tillis Delays Clarity Act Stablecoin Yield Text, Citing Uncertainty Over Markup Timing” (April 2026). Describes the central dispute: bank trade associations pressing to bar exchanges from paying yield on stablecoin balances, after the GENIUS Act had already barred issuers from paying interest. Also gives the House vote.
12. AMINA Bank, “CLARITY Act September 2026: Why the Senate vote failed and what regulates US crypto now” (September 2026). Reports the 49–50 cloture vote of September 15, 2026, and attributes the failure to two disputes: presidential conflicts of interest in digital assets, and community banks’ objections to stablecoin rewards. For the timeline, including the January 2026 markup postponement over stablecoin yield and the terms of the May compromise, see Datawallet, “CLARITY Act Explained,” and Sumsub, “What Is the CLARITY Act?”
13. The Startup Law Blog, “Accredited Investor Definition (2026): Income, Net Worth, & Verification for Startups” (verified August 2026). Under SEC Regulation D, Rule 501(a), an individual qualifies with income over $200,000 ($300,000 with a spouse) or net worth over $1 million excluding the primary residence, or by holding certain securities licenses. The SEC has listed possible amendments on its agenda but has proposed none.
14. LawCast, “SEC and NASAA Statements on ICOs and More Enforcement Proceedings” (January 16, 2018). Summarizes the SEC’s July 2017 report on The DAO and Chairman Jay Clayton’s statement of December 11, 2017.
15. Andrew Kolodny, “How FDA Failures Contributed to the Opioid Crisis,” AMA Journal of Ethics 22(8), August 2020. Source for the approval evidence, the reviewers’ later employment, the 2002 advisory committee, the private meetings on trial design, and the absence of a root-cause analysis. Kolodny has served as an expert witness for states and counties suing opioid makers.
16. Centers for Disease Control and Prevention, “Understanding the Opioid Overdose Epidemic” (updated August 28, 2026). The 806,000 figure covers 1999–2023 and includes deaths involving both prescription and illegal opioids.
17. US Department of Justice, “Opioid Manufacturer Purdue Pharma Pleads Guilty to Fraud and Kickback Conspiracies” (November 24, 2020).
18. Olivier J. Wouters, Martin McKee and Jeroen Luyten, “Estimated Research and Development Investment Needed to Bring a New Medicine to Market, 2009-2018,” JAMA 323(9), 2020. Median $985 million and mean $1.3 billion per approved drug, counting the cost of failed trials; estimates from industry-linked studies run higher.
19. Jack W. Scannell, Alex Blanckley, Helen Boldon and Brian Warrington, “Diagnosing the decline in pharmaceutical R&D efficiency,” Nature Reviews Drug Discovery 11 (2012). The “Eroom’s Law” paper. Scannell has since said the figure has been roughly flat since 2010.
20. US Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation, issue brief on FDA user fees. In fiscal 2022 user fees were 66 percent of the human drugs program budget and 46 percent of the FDA’s total budget.
21. US Department of Justice, press release on the joint DOJ–FTC statement on Virginia’s certificate-of-need laws (October 2015). The agencies say such laws raise barriers to entry, have stifled innovation, and can be used by incumbents to stop or delay new competitors.
22. Becker’s ASC Review, “The 29 states where competitors can veto a CON” (August 17, 2026). Summarizes a Cato Institute briefing paper by Stephen Slivinski and Matthew Mitchell; Cato favors repeal of these laws.
23. Wilk v. American Medical Association, 895 F.2d 352 (7th Cir. 1990), affirming the 1987 district court ruling. The district court found that the AMA had violated Section 1 of the Sherman Act through a boycott meant to contain and eliminate chiropractic, while accepting that concern for patient care was its dominant motive. The AMA’s Committee on Quackery was formed in 1963.
24. RTHK, “HA expands Chinese-Western medicine services” (May 5, 2025); Hong Kong Baptist University, “Launch of Chinese medicine hospital opens new chapter in healthcare for Hong Kong” (January 30, 2026).
25. Nobel Assembly at Karolinska Institutet, press release on the 2015 Nobel Prize in Physiology or Medicine (October 5, 2015). Tu Youyou shared the prize for the discovery of artemisinin, which the release traces to a screen of traditional herbal remedies and to clues in the ancient literature.
26. US Food and Drug Administration, “What is a Botanical Drug?” Lists four botanical products approved to date. The figure of more than 800 applications and pre-application meeting requests before 2018 is from FDA staff, “Scientific and Regulatory Approach to Botanical Drug Development: A U.S. FDA Perspective,” Journal of Natural Products 83(2), 2020.
27. SupplySide Supplement Journal, “Botanical Drug Market Hampered by Regulations” (December 28, 2012).
28. National Center for Complementary and Integrative Health, “NCCIH Funding: Appropriations History.” $170.3 million in fiscal 2024, against a total NIH budget of roughly $47 billion.
29. Wisconsin Watch, “Did drug companies spend $10 billion on consumer advertising in 2024, making up nearly 25% of evening ad minutes?” (July 24, 2025). A fact brief: 24.4 percent of evening ad minutes on news programs on ABC, CBS, CNN, Fox News, MSNBC and NBC from January 1 to May 31, 2025.
30. OpenAI, “OpenAI acquires TBPN” (April 2, 2026). The acquisition announcement specifies the show’s place in OpenAI’s strategy organization and promises continued editorial independence.
31. OpenAI, “A landmark multi-year global partnership with News Corp” (May 22, 2024). Documents a commercial relationship involving publisher content. It does not establish interference with editorial decisions.
32. Tarbell Center for AI Journalism, “About / Our Supporters.” Publicly identifies its institutional donors, including Coefficient Giving, the Survival and Flourishing Fund, and the Future of Life Institute.
33. Tarbell, “Tarbell Launches $10 Million Institutional Grants Program to Expand AI Journalism” (October 5, 2026). Announces the goal of distributing at least $10 million by the end of 2027. This is a program target; it does not say the entire amount has already been paid.
34. Tarbell, “Editorial Independence.” Sets out protections against donor direction of content, framing, conclusions, and the selection of fellows or stories. These commitments are a separate question from ecosystem-level diversity.
35. Anthropic, “Anthropic is donating another $20 million to Public First Action” (July 21, 2026). States that total support reached $40 million and that the funds are restricted to public education and policy work and may not be used to influence candidate elections.
36. Reuters Institute, “An Industry-Led Debate: How UK Media Cover Artificial Intelligence” (2018). An eight-month study of six UK outlets. Given its scope and age, it supports concern about industry-led framing but is no universal finding about present-day media.
37. Hanna Barakat, “Interrogating Mainstream Reporting on Artificial Intelligence,” Tech Policy Press (August 12, 2024). Reports a mixed-methods case study of New York Times coverage from January–March 2024, including the finding that 67 percent of quoted sources worked in commercial technology.
38. Rasmus Kleis Nielsen, “How news coverage, often uncritical, helps build up the AI hype,” Reuters Institute (May 20, 2024). Reviews research indicating that coverage is often positive and industry-led. This qualifies any claim that catastrophic-risk advocates dominate the entire media landscape.
39. Amrith Ramkumar, Erin Woo, Berber Jin and Ben Cohen, “The Anonymous Math Geek Who Quit Anthropic—and Became the Face of AI Safety,” The Wall Street Journal (September 17, 2026), as republished by To Vima. Source for Coxon’s background, the announcement of his resignation, the lab leaders’ joint call to slow down, and Trump’s response. Coxon says he decided to leave on his own and did not coordinate the social-media response with anyone at Anthropic. See also my September 15, 2026 post on Coxon.
40. Stephen M. Omohundro, “The Basic AI Drives,” Proceedings of the First AGI Conference (2008). The original argument concerning self-improvement, rationality, goal preservation, resistance to counterfeit utility, self-protection, and resource acquisition. The paper also recognizes that goals concerning others’ welfare can alter resource-seeking behavior.
41. Ben Goertzel, Basic AI Drives, five-part series (2026). Especially Part 2, “Which Minds Have Beneficial ‘Basic AI Drives’?”; Part 4, “How Big Are the Basins Around the ‘Basic AI Drives’?”; and Part 5, “Simulating the Basins Around the ‘Basic AI Drives’.” The arguments are conditional; the toy simulation does not supply measured probabilities for real AGI populations.
42. Ben Goertzel, “The Nine Days of Madness” (October 9, 2026). Fiction illustrating different developmental trajectories, including both beneficial cooperation and the hazards of recklessly extending an open experiment. Not offered as empirical evidence.
43. MIRI Technical Governance Team, “An International Agreement to Prevent the Premature Creation of Artificial Superintelligence.” A concrete treaty proposal involving US–China coordination, training restrictions, monitoring, and verification. Its existence is a separate question from the political feasibility of securing and enforcing it.