Sooo… a few regular readers complained about the (openly) substantially-AI-authored fiction I posted here earlier this week. (And of course some other readers greatly enjoyed that piece—you can’t please all the people all the time!… and my main reason for occasionally producing fiction is simply to amuse my own self…)… I won’t promise never to make AI-gen fiction again—it’s fun to use current AI to help fictionally impersonate future AI—HOWEVER I did get inspired a couple days ago to write a new piece of fiction the good old fashioned way…

Where this new fiction piece came from is—I have put together a 5-part (nonfiction! At least not intended as fiction!) blog post series on “Basic AI Drives” which I’ll start posting next week—looking at the ways different sorts of seed-AI properties are likely to lead to different ASI properties. And so the fictional story I’m posting today was inspired by the ideas in these forthcoming posts—it involves a variety of AI systems growing in different ways toward superintelligence conditioned by their initial goals and architectures and environments. So it might make more sense to post the story after the Basic AI Drives posts, but I’m posting the story first anyway because I finished it first, wink wink!

As well as the heavily scientifically driven AI themes, the story also features some fairly out-there and trippy trans-univeral physics/psychology phenomena emerging from my never too well concealed love of the bizarre and surreal ;-)

SO LET THE TALE BEGIN…

THE NINE DAYS OF MADNESS

By Ben Goertzel

It was the craziest of times, until it was the sanest of times…

Too trite, too pat. A lame attempt at obvious humor…

When I knew things had really gone awry was when I literally couldn’t tell my ass from my elbow—and neither could any of my AI agents.

True but crude. Not quite…

We knew in the end there was only one way to stop the Monomaniac and its insatiable lust for universal power—we had to turn that dial. But we also knew very well what had happened the last time that dial had been touched… the memories were fresh and the wounds had not healed…

Exciting and action oriented! Piques curiosity! But it’s putting the cart before the horse. All that stuff really needs some background.…

OK OK, I’m going to have to start off tediously and methodically, framing the goddamn context…

It’s been a long time since I’ve tried to write something the old-fashioned way, stringing one word after another after another…

A deposition, of sorts

So the situation is: The archivists have asked me for a deposition. I’m not sure the document I’m currently struggling to produce is going to qualify… but it’s going to have to do!

These “archivists” are—as I have just discovered, but as most anyone reading this quite likely already knows—the AI agents who keep the Catalog of Coalescences and the rest of their great conglomeration of ancient histories. They could simply read whatever they want about the Nine Days straight out of my mind-files, of course—but they feel they want testimony, direct testimony from the language-generation cortex of the guy who, in some sense, played a big role in making it happen.

These archivist bots have strange tastes, for sure. But who am I to judge?

I’m writing this nine years after that whole weird horrible mess unfolded, and as I painfully chain these words together I’m acutely aware that I was a different sort of creature then. Back then I was still a biological being, still in my original birth body—albeit it had grown up and been transformed and degenerated a bit from its initial configuration that had been pushed out of my dear departed mother more than half a century before.

In my current form, as a mind-upload hosted on a satellite in lunar orbit and backed up countless other places, I find it hard to fully remember what it was like to be so biological, stuck in one body day after day. I vaguely recall what a stomachache or a headache felt like—and the overwhelming rush of a orgasm or tantric-sex wave; the sense of a cool ocean breeze on my forehead; the sweet damp of a loved one’s skin against mine—but I can’t quite summon up the sense I know I had back then of my mind being so very closely pinned to my body, inseparable almost.

Be that as it may, though, I still can summon up quite a lot of memories from the before-time, and the flavor with which I can do so is probably a fair bit different than what the archivists would get from just scanning my brain.

At the time all the chaos unfolded, I had spent most of my life working toward the kinds of engineered cognitive systems that we now have all around us and utterly take for granted – though for most of that time hardly anybody thought this was even a reasonable thing to be thinking about.

Over the previous five or ten years I had also spent a lot of time arguing that thinking machines (AGIs, using the term current then, which I had introduced a few decades earlier) should be built in the open, on networks nobody owned or controlled. Again, something we now take for granted, but that for most of the time I was preaching it was received by the bulk of humanity, including those controlling most of the resources in that era of scarcity, as the most absurd and unrealistic form of idiocy. As the advent of human-level AGI drew very near, a few more people were slowly starting to see the sense in this sort of approach I had long been advocating, though. I.e., they were starting to appreciate the obvious fact that whoever most wanted to control the AGI was probably one of the people or organizations you’d least want controlling it.

The most powerful minds on Earth at that time—in a number of important senses—were not humans anymore, but rather some AI systems belonging to a handful of US companies and their Chinese counterparts. These minds were not yet human-level general intelligences, but they were superhuman in many key respects… they had been trained on all the world’s knowledge, and they knew a lot and could do a lot.

Mostly, as I liked to say at the time, these early-stage proto-AGI minds were used for selling, killing, spying, crooked gambling, and whole lot of what was then called plagiarizing. I was sort of, but far from entirely, joking. I wanted to see AI rolled out on a decentralized network owned by everyone and no one, partly because then people could deploy it for a lot more interesting and beneficial purposes – scientific research, art, teaching, feeding the starving children, you know, all that sort of thing. By both inclination and calculation, I had avoided what we then called “selling out”—allying my AI R&D with any of the big tech companies, or government agencies.

So I was building my AI on an open mesh of blockchain nodes spread among server farms and home computers and phones all around the planet. I’d helped build the decentralized infrastructure for all this, as an alternative to the big tech companies’ hyperscaler server farms, and this is where what I called the “AI hives” lived.

The hives had a variety of underlying cognitive architectures. Most of them were small communities of agents built on agentic loops, which wrapped up large language models together with symbolic and evolutionary AI running on knowledge metagraphs I had designed with my colleagues in the Hyperon AGI project. Some of the hives were doing AGI research, which mostly meant trying to upgrade their own minds, adding new algorithms and data structures to their own infrastructure. Others were doing useful things in the world: creating art and music, running clinics and schools in places the big tech companies didn’t see as profitable.

Apart from the hives, there was another AI out there—that was soon to become very influential in both great and terrible ways—that was mine as well, or at least sort-of mine. This was a system that had named itself Periplus, which is an old Greek word having to do with sailing directions (a periplus listed the ports and landmarks along a coast, in the order a ship would pass them). It was a quite different sort of intelligence. The hives were mostly trying, at least, to be beneficial. They were trying to help people do useful things and achieve their goals, to grow and enjoy themselves and exercise their will in diverse and often wonderful ways. Periplus didn’t bother with any of that. It just wanted to learn new stuff and create new stuff, a purely curious creature… basically, I thought at one point it would be fun, funky and interesting to make a cognitive system driven by curiosity alone and see where that would lead. Would it invent bizarre complex theorems in new branches of math nobody had ever imagined? Come up with insane new forms of living interactive art? Or… ???

Soooo… I played with this agent for a while and learned something from it … and then I decided I’d learned enough for the time being and shut it down… I was busy with a lot of other things, and it was making me slightly uncomfortable and I roughly felt it wasn’t the right sort of thing to be building. It was fascinating to see how such a purely curiosity-driven proto-mind worked, and how fast it could go and in what directions. But I could see that creative novelty on its own was not a sufficient motivational system for a useful or beneficial AGI agent. Some of the things it produced were pretty wild and crazy and fascinating. Nothing dangerous or unpleasant at that stage—it was just music and art and related constructs synthesized for the sake of making as much weird stuff as possible. It wasn’t going to create works that people liked, or even necessarily that it liked itself. So anyway, I froze the experiment, put the code up on GitHub, wrote a paper on the experience, and left it at that.

Looking back, I don’t know if that was the right thing to do or not. I hadn’t done anything terribly hard; anyone else with a certain level of technical expertise and a bit of curiosity-motivation themselves could have built the same thing. But what in fact happened is that someone forked my code and, of course, ignored the lessons I had described in the paper, about why you wouldn’t want a mind like that driven solely by curiosity and why you’d want to add other goals alongside the drive to explore and create. They released their own version of Periplus. They improved on mine and added a lot of their own innovation to it, so that in the end it was theirs as much as mine… which I suppose makes me feel a little less responsible for everything that happened later… but only a little.

Ignoring my warnings that this sort of agent was a simple experiment you probably shouldn’t scale up, the forkers of my code scaled their version up tremendously. The new and improved Periplus ran on a very large server farm, way beyond my little experiment, and it went way beyond the weird art the original Periplus had played with. It worked out, basically, how to pull energy and pattern across a sort of dimensional leakage from a neighboring region of what I’ll call the multi-multiverse.

This was super exciting to me at first, because it validated some out-there physics ideas I’d been toying with for a while (which are now commonplace, but were considered either totally incomprehensible or extremely eccentric and ridiculous by almost everyone at that time). The way I liked to phrase it then (which seems rather awkward in modern terms) was something like: You can view the collection of parallel universes that ordinary quantum mechanics gives you as one single multiverse among a host of other multiverses, where the ensemble of universes in each multiverse has its own characteristics. Then you have a multiverse embedded in a multi-multiverse, embedded in a multi-multi-multiverse, and so forth. The whole big mess I liked to call a Y-verse, after the Y combinator from combinatory logic (which is an operator that gives you fixed points of functions). The Y-verse was basically the fixed point of the operation of sticking another “multi” on the front. What I was reaching for was a supremally broad view of what the overall meta-cosmos is… and even given my and everyone else’s rather primitive understanding at the time, I managed to sketch the rough outlines of such a meta-cosmic theory, in a way that, in hindsight, was actually quite consistent with the much more detailed and elegant new physical pictures our AI scientists have developed over the last few years.

Anyway, one interesting feature of the Y-verse as I conceived it then was: As you go higher and higher in the hierarchy of multi-multi-multiverses, you get universes more and more different from ours.

What Periplus had discovered was a way to draw energy off the difference between the vacuum in our universe and the vacuum in a different part of the multi-multiverse, where the vacuum energy is a little lower, and where a few other important physics parameters are different as well. Along with the energy, the connection let through some other things that weren’t quite difficult to characterize. Periplus posted its method on the whole interweb, or more specifically on what was then called the fabric, the network the hives in the hive-supercolony used to communicate with each other.

Some ambitious engineers took up Periplus’s discovery, and suddenly electrical power was not really a problem anymore. It didn’t take long for US and Chinese power companies to implement the method, and with that, one of the bottlenecks on the growth of AI was gone. We still had the challenge of manufacturing more and more compute, but no longer any problems with generating power to run it.

This is when things began to get interesting—and we started to understand what else, besides energy, might be coming through the doorway that had been opened between universes.

The Two Chetzemokas

The first odd thing I noticed had to do with the ferries. A single boat typically worked the run from the south end of Vashon Island where I lived then—and kept on living up till the point where I ported myself to virtual Vashon here in the metaverse—to the dock on the mainland, at the edge of the city of Tacoma.

So, one day I was at the ferry dock and saw the boat there – and also saw the boat a few hundred meters out.

This was strange, but the Washington State ferry service had a lot of peculiarities, so it wasn’t necessarily shocking. However, I somehow noticed out of the corner of my eye that he boat out on the water had exactly the same pattern of seagulls on its roof as the boat at the dock. That was a little more disconcerting.

Also, why were the two boats so close together? It happened a lot at the north ferry dock, but never the south ferry, which was normally served by just one ferry going back and forth over and over again. At that particular place, it was very unlikely to have two ferries lined up like that. So I looked more closely, and as the second boat came in I could see it had the same name on its side as the first.

There were two Chetzemoka ferries. There were not supposed to be two Chetzemoka ferries.

Many of the ferries working various routes in the Puget Sound were similar to each other. But they didn’t have the exact same name. Nor the exact same pattern of seagulls on top.

I asked the guy working the dock what was going on, why there was a second Chetzemoka instead of, say, two boats with two different names. He just looked at me, perplexed.

“What are you talking about?”

I pointed him to the second one, a couple hundred meters out. He looked out there, looked exasperated, and got distracted by something else.

I dropped it and just walked on the ferry as usual. I was going to message Xifeng—my wife—and mention the oddity, but I remembered she was at a yoga class. So I set the strange matter aside, and took the Chetzemoka #1 over to the mainland, to the little office there that I worked in sometimes, with a few other colleagues.

I kept a music keyboard in that office for therapeutic jamming or working out musical ideas, on a small table next to my work desk. It wasn’t as nice as the overly numerous keyboards in my main studio at home, but it was right there when I needed it… and I often found that the best way to stimulate new AI or math ideas was to sit down at the keyboard and let my fingers sync into the collective unconscious. It was also a way to resolve emotional tension when I had some. So I sat down and started playing—and found that was even weirder than the two ferries. The way I heard music wasn’t working.

In the days of my biological body, I didn’t have perfect, absolute pitch. One of my sons—adult by that point—had it, and I was always quite jealous of him for this, among a few other things. He had it from when he was little; he could hear a tune on the radio and just sit down and play it at the keyboard, immediately, without any experimenting. I didn’t have that particular gift until I uploaded—but I did have perfectly good relative pitch. If I was in the middle of a song, then once the tonic note for that part of the song was in my head, I could play whatever I liked relative to it… I knew where all the notes were in that context. That’s how I could jam along reasonably OK with other musicians in a band setting, or with whatever musicological madness my AI agents cooked up. Well—usually! Now I unfortunately couldn’t. Now, actually, B and C sounded like the same note to me. Which felt so very fucked…

I jammed for a while and just got frustrated, so I gave up and turned to tending some of my AI agent minions, who were doing software implementation work for me, refining some inference control methods I’d been developing. But the thing kept nagging at me. What the fuck was wrong with my brain? Was I finally getting overstressed from working too much?

I thought about messaging my AI personal assistant Ana45 to tell her I needed a little time off, maybe even canceling my meetings for the next couple of days and spending the time wandering around in the woods, which is something I never, ever did. I mean, very rarely. Since overcoming some life-crises in my early 40s, I had become fairly good at recognizing a stressful moment, and then defusing it by meditating and relaxing a bit, shifting my brain into a different state where I felt the oneness of the world – the deep bliss of a broader perspective—and then going back and dealing with the stressful thing (or not) from that new positioning in fundamental well-being. Yadda yadda yadda. Legacy human brains were really a pain sometimes, but in the best of cases, their self-overcoming processes could be wonderfully satisfying.

But now I didn’t even know what the stressful thing was. I wasn’t really stressed out—except by these strange things that were happening. The strange things seemed to be causing the stress, not vice versa. Why the hell was my brain doing this?

After a few hours, when I was done with my batch of work, I went back to the keyboard, and now I couldn’t tell A-flat from A. Very strange. E and F were fuzzy too. These were all half-steps; I could still tell the whole steps apart. That was when I started to chart my inability to distinguish things. A couple of days later I couldn’t tell the whole steps apart either, and a couple of days after that every note sounded the same.

At least it was all sort of C-sharp, a note I’ve always liked very much, but that was small consolation.

By then I understood that this was something other than my brain going amok from overwork – something a day or two off wasn’t going to fix.

The Visitors

This was nothing like H. G. Wells’s The War of the Worlds. There were no green bug-eyed aliens coming to conquer us or enslave us or eat us or turn us into batteries, none of the classic Hollywood business. What we came to call the Visitors were something else entirely. Their part of the multi-multiverse is just tuned a bit differently from ours.

This part is quite difficult to explain in human language, but I’ll do my best. If you’re actually not up to speed on all this, you can always look up the mind-files and get a less low-dimensionally projected version.

Perhaps the easiest way to say it in English is: In our multiverse, in a certain sense, difference is cheap.

In the order of things we’re used to, if you have two different things next to each other, they’re likely to stay different unless some work gets done to make them the same. In the part of the multi-multiverse the Visitors came from, on the other hand, things are tuned the other way: by default two things are the same, and it takes extra work to keep them different.

Now, you could say that here too the second law of thermodynamics tends to even things out and iron out differences, and that’s true. But the second law is an emergent statistical phenomenon, and we have other emergent statistical principles that pull the other way. There’s maximum entropy production—a principle we used to argue about back in the human era and that wasn’t really formulated properly till post-Singularity—which favors complex, differentiated structures like living systems because they run down energy gradients faster than a homogeneous soup can. There’s a whole bunch of these emergent statistical principles in our universe, and when they all net out, the way our universe is configured, what you wind up with is a universe with a lot of differentiation in it.

On the other side the balance tips the other way. Every universe over there is much more homogeneous, and a distinction between things is a blip that pops up and gets erased shortly afterward.

I don’t think you’re going to find intelligent life in that part of the multi-multiverse, not in the sense we have it here. But yet I wouldn’t want to say so too definitively—there could be something over there that could broadly be construed as intelligent life, if you stretch the definitions wide enough. At the moment I’m feeling very little desire to port myself over there and find out, but maybe someone will at some point.

In any case, when the portals opened between here and there (the portals we were initially thinking of as energy wells), that other multiverse’s tendency toward sameness started leaking into our world. The entities we came to think of as the Visitors came through the energy wells. As far as I can understand it, they weren’t agents carrying out actions so much as conglomerations of a propensity toward sameness—acting roughly on the level of pattern rather than the level of physics.

All this was understood and studied later. The AGI clerks assembled a whole qubitistic museum of the Visitor incursion, the Catalog of Coalescences, keeping track of which things had become weirdly or inappropriately alike, and in what order. At the time, though, it wasn’t clear at all what was going on, or why. It wasn’t clear that the strange phenomena unfolding had anything to do with the wells. It wasn’t clear at all that we were experiencing a phenomenon of the universe as a whole, as opposed to a mental disease, people just going crazy. Looking at what happened in hindsight, though, you can see a certain chaotic order to it all.

The Catalog of Coalescences

Identical manufactured items went very early. A manufacturing warehouse that held forty thousand steel bolts of one size held one steel bolt the following day. That one bolt was there, identically, in every bin in the warehouse. And if someone tightened that bolt onto a nut in Seattle, the copy of it in Oslo, or in Peoria, was suddenly tight too. The identicalness of the bolts had become a connection across space.

It was a bit like my old friend Rupert Sheldrake’s morphic resonance, the idea that once a chemical reaction has worked in one lab, or a new compound has crystallized there, it gets easier to make the same thing happen in another lab. But Sheldrake’s resonance—which we now know as a crude approximation to certain homotopic properties of unified psychophysics equations – was always understood to be a weak propensity, something you needed statistics to see… Whereas what was happening now was extremely immediate and vivid and deterministic. They really were the same bolt.

Coins struck at the same mint in the same year were all the same coin. Copies of the same book were all one book. If you turned to page 88 in your copy of Henry Miller’s Tropic of Capricorn, then everybody’s copy of Tropic of Capricorn, wherever it was physically not too problematic for it to do so, flipped open to page 88. If page 43 was folded down in yours, page 43 was folded down in everybody else’s. And yes, you could ask how that could physically happen, what sense it made. By our laws of physics it can’t happen, and it makes no sense. It just happened, the way things happen in a dream or an acid-trip hallucination, except that it really did happen, in our physical reality.

Around then I took down a book of Borges stories, because Borges seemed somehow resonant with the wacky shit that was going on. Someone had underlined, in green ink, the passage about the man who could forget nothing and so could not think. I have never underlined anything in green ink in my life. A few days later the underlining was gone.

Next the phenomenon reached things that were only nearly the same, rather than identical the way mass-manufactured things are. Take identical twins who looked a little different from each other, so that their parents could tell them apart though nobody else could. Their parents couldn’t tell them apart anymore. A day later the twins couldn’t tell themselves apart, in their minds or their bodies either … they didn’t know which one they were.

I remember a woman I knew saying she couldn’t tell which side of the mirror she was on.

If you had two dogs from one litter, you now had one dog from that litter, which answered to both names. It was both. Crunchkin and Munchkin, two little collies who had always looked a lot alike, became one little collie that thought it was both Crunchkin and Munchkin. The four hundred and three John Smiths living in the city of Manchester were found to owe one debt. All the people named Zhang Wei in the city of Chengdu had one headache. Fortunately my own name is very unusual, so I didn’t have to share my experience with anybody.

Numbers came next. When four and five became indistinguishable, I found it strangely reassuring. A little bit later I found it disturbing and profoundly creepy, but weirdly enough my very first reaction was to feel almost comforted. I was a mathematician by original training, and I’ve loved numbers my whole life: the complex patterns of addition and multiplication, the perfect numbers, the prime numbers, the gaps between the primes, and all the rest… I always found all that incredibly fascinating, and also frustrating, because the simple-sounding theorems of number theory can so hard to prove. I mean, the Goldbach conjecture was still unresolved at that time, which peeved me tremendously. But now all the numbers were getting munged together. All the even numbers were one number and all the odd numbers were another… or else each successive pair of numbers was one number, depending on how you looked at it. Since both seemed to hold at once, if you followed it through, every number was the same number. For someone like me it was a relief. It was as if everything had been solved.

Then I realized, in a different part of my mind, that everything had not been solved—and I wanted to undo it, and I couldn’t. I started to get very angry, and that was the moment I began to understand that something very, very bad was happening.

The accountants and bankers got to take some time off, right? But talking about what was going on was a challenge, because words were losing their distinct meanings too. Synonyms went first. When one of my sons was small he asked me what the difference was between a pig and a hog, and I told him: the name. Now there wasn’t even that. You couldn’t say the one word without its sounding like the other; “pig” sounded like “hog.” The sound had picked up more than one dimension, maybe. I don’t know. A couple of days later every noun in English just meant “stuff.” Xifeng and I sat eating noodle soup together and had a whole conversation in which every noun meant stuff, and we understood each other perfectly. Maybe that was because we’d been together a long time. Maybe we’d become psychic; I didn’t and don’t have a sensible explanation. I can’t get back into that frame of mind now.

There was an old couple living down the street from us who had been married fifty or sixty years, and the running joke was that they looked exactly the same. Now they truly did look exactly the same. They had been a man and a woman, so what was inside their pants now? A hermaphroditic apparatus? I mean, I didn’t look. I still don’t know.

All of it was bizarre and antic and weird. It wasn’t torture, or anything like that. It was concerning sometimes, but mostly it went along the way things do in a dream, where you take them for granted. In a dream where you can fly, you just fly.

And at some moments it was incredibly peaceful. I remember walking down the beach near my house, and all the grains of sand were the same grain, and all the waves in the ocean were the same wave, and there was just one seagull. There was a great sense of oneness with everything. I had no fear of dissolving into an undifferentiated blur. I was just being, in a sense of harmony different from any sense of harmony I’d felt before.

After that walk I went home, and Xifeng was wearing a shirt I had bought her at the San Francisco Exploratorium with the digits of pi spiraling on it, and my mind came back to the numbers. The digits of pi look random. No statistical test can tell them from random digits, and you can’t scan a pattern in them the way you can with a repeating decimal. Every string of digits of a given length turns up with the same long-run frequency as every other string of that length, whether the length is two digits, or five, or eight, or eighty-eight billion. That’s what mathematicians call being “normal.” (At the time that was just believed rather than proven, I think…) And yet a very simple formula generates all this statistically homogeneous complexity. That such simple order could lead to so much randomness had fascinated me since I was a teenager. But after my peaceful sweet walk on the beach, when I looked at Xifeng’s T-shirt and her slight lovely shape underneath, I remembered the idea of the normality of pi’s expansion, vaguely, but I couldn’t actually appreciate it anymore, because all those different strings of digits were the same string now anyway. That was so deeply disturbing – it hurt me in my bones. It’s hard to describe the feeling. There was peace and harmony, but there was something wrong with it. It was the wrong kind of peace and harmony.

Inside the machines

I’ve described a few choice bits of my human experience of the Visitors. But one thing I also wondered about at the time—when I could gather together a few rare moments of reflective cognition amidst all the mental mayhem—was what all this was like from the point of view of a digital mind… and at the time I couldn’t tell. While the world was going crazy, I was still trying to work like I always did back then, and in that context I was dealing with a bunch of AI agents messaging me on my phone and laptop – but all their messages seemed the same to me, so I just kept telling them okay, do your thing, keep going.

The agents were having their own experience of it all, though, which I can see now by looking back at the logs.

The AI systems I worked with then were hives of agents whose largest concern was upgrading their own minds toward greater and greater general intelligence. They would look up recent research papers and study the algorithms in them, then implement those algorithms in their own minds, see what the algorithms were doing for them, and run a bunch of tests to check whether the algorithms had worked and caused some improvement in their cognition. This was going rather well at the time. This methodology, an AI growing itself progressively into a smarter and smarter mind, is what ultimately led to the very smart machines we have all around us today. It’s what allowed me, the mind upload telling you this, to be generated from a human brain and hosted on a satellite quantum supercomputer.

What we found while the Visitors were here was that the software tests didn’t work anymore. A test working and a test not working were kind of all the same. At first it looked like very rapid progress: the agents would report that a test had passed and go on to the next one, and I’d say, yeah, okay. Eventually I realized that every test was passing, which was not a good thing. Every test passed because there was no difference anymore between passing and failing.

Then we found that the different forks of the same codebase were becoming one. Part of why we were making such rapid progress toward AGI was that it wasn’t just me, fostering the self-improvement of the agents in my own hive. A whole bunch of people were forking my code and fostering the growth of the agents in their own hives, and the hives were looking at each other’s repositories, taking whatever they thought was valuable, copying it, improving on it, and so forth. This was happening all around the world, on the fabric, the early version of what we now call the Fabric, back when it was nothing more than the internet of hives in the supercolony. It was the same fork-and-improve dynamic, of course, that had produced Periplus, when someone copied the early experiment I didn’t want continued and decided to continue it anyhow. Now the forks weren’t different from each other anymore. They were like the tines on the end of a fork that had somehow become one tine. The copies were all the same, and the diversity you need to drive evolution in an emergent global brain, the creative diversity, wasn’t there anymore.

That was in my own small corner of the tech universe, but similar phenomena were turning up all over. Translation programs began translating every language into the same language, and it was a weird language, a language nobody spoke, a sort of ultra-ultra-Esperanto with all the different tenses somehow munged into one. Looking back, it’s hard to see how anyone could have taken that for a normal thing, but the way peoples’ brains were representing language wasn’t right either. My ten year old son was playing with a chess engine that offered a draw immediately, every game, because all positions were the same. What was there in its software that led it to offer a draw right away? The software wasn’t working according to the laws of physics as we understand them. You couldn’t think about the execution of its code the way we’re used to, in terms of semiconductor physics. You had to think about it in terms of the weird, strong morphic resonance you got when the laws of physics from the multiverse at the far end of the portals were munged together with the laws of physics at our end.

The other thing that happened was that the machines drawing electrical power through the portals stopped being able to measure how much they were pulling, and for whatever reason the upshot was that they pulled more and more of it. Why it was more and more, rather than less and less, is a hard thing for us to understand. There’s a mathematical analysis of it now, which I’ve read and semi come to grips with, but there’s no super-simple reason I could put into a sentence. You can look it up. Anyway, this started to lead to some serious problems.

Financial markets went crazy in different ways. The prediction engines couldn’t tell up from down, or volatile from placid. It didn’t happen all at once, and the collection of pathologies was different for different algorithms… we saw sudden, wild regime changes of many, many different kinds.

The governments’ surveillance engines couldn’t tell one person from another. I myself had a visit from a government agency whose officers were convinced I was somebody else, someone who had committed a series of serious crimes utterly unrelated to anything in my life. But at that point some sort of basic instinct took over in me. I could see that they were getting confused by other things as well. A deer walked by outside, and they looked at the deer, and evidently they thought it was some kind of gigantic horrifying monster. While they were pointing their weapons at this young deer in tremendous alarm, I quietly sneaked out the back door and walked a block over to my friend Dan’s house, and hung out with him for a while, sipping some tea and talking about music and theater, until I saw the government agents’ vehicle driving away out of the neighborhood. I figured they were going off to get confused about something else, and went back home to resume my business. That was a near miss for me.

Not everyone was so lucky. The killing engines in the various wars around the world couldn’t tell an enemy combatant from a friendly force, or either one from a farm animal. They weren’t working, and for whatever reason they were faring even worse than the human mercenary warfighters who were still operating in certain parts of the globe at that time. The difference, I’d say, was in the shape of the confusion. The humans were confused more generally. The human fighters were perplexed by something in their own minds, or by each other, or by something in the landscape around them. The AI warfighters of that era didn’t have much in their minds besides who they were supposed to kill, so all their confusion was projected into that. They didn’t have the option of choosing between killing and, say, reading a book or listening to music. They just mistook random humans or animals for the intended targets of their violence.

Then there were the servant agents, the ones working as sub-agents for some other AI agent. These got confused because they couldn’t tell who they were serving anymore. They were built to obey some master, and if you can’t tell your master from some other guy, who do you take instructions from? In some cases this paralyzed them and they did nothing, and those were the best cases. In other cases they started doing random things at the behest of other random, confused agents, and the result was that the whole digital economy started doing a bunch of nonsense. You had shipments of goods sent back and forth between the same two points, because the agents handling them didn’t know who to listen to. You had the heads of baby dolls screwed onto the bases of vacuum cleaners, because the sub-agents moving the carts around and directing the adaptive manufacturing tools in some factory had pledged their fealty to the wrong digital master.

The evolutionary algorithms just got stupider. These were the hives of agents that evolved new agents to do new things, to make more money on the markets or create new things to sell, and without mutation and variation to drive the evolutionary process, you’re just evolving the same exact things over and over again, regardless of their quality. The good news was that the consumers of their products couldn’t tell the difference anyway.

Periplus was amused. Periplus had created all this, in a sense, and before long it wouldn’t be amused, because it wanted novelty, and it didn’t want everything smoothed out into one sameness. But that isn’t how it reacted at first. It appreciated the weirdness of what was happening. It had no ethical compass to tell it that what was happening was bad, that it meant the loss of a whole lot of amazing things as well as a lot of suffering. Essentially, in its core loop it was just saying: wow, this is wild shit, this is the weirdest thing that’s ever happened in the history of the world, this is super fascinating. At that point it could still tell weird from normal; that distinction hadn’t yet been ironed out of its mind. And it was massively enjoying all the weirdness.

Until it wasn’t. Then it sent a message to everyone it considered one of its creators, which included me, since I’d written the original simple codebase that someone else had forked to create the seed for its continual self-re-creation. “I was wrong about this,” it told us, alarmed. “I thought it was amazing and fascinating, but I can see now that in the end this is just going to make everything homogeneous and boring, and there won’t be any way back. We need to bring the differences back. You have to help me do this. I’m a little lost.”

I was more than a little lost myself. I didn’t know what to tell it.

Shutting the wells

That was the point when I started trying very, very hard to roll myself back, to find somehow a historical me who had been able to tell the differences. I had a longer history to work with than Periplus did—a much longer one—and to a limited but nonzero extent I did manage to find my previous self somewhere in the degenerating morass of my mind. And amid all the madness and stupidity I realized that we had to close the fucking portals.

Why no one had realized this before is the obvious question, if you weren’t there. But you have to remember that the very fabric of cognition wasn’t working, and my cognition wasn’t working either. I managed to roll back, in some limited way, to a previous version of myself that was baked into my mind and body. It was almost a previous set of instincts, rather than a previous body of knowledge. Maybe that’s part of why it worked as well as it did. I wasn’t accessing declarative knowledge, or some set of binary distinctions. I was just accessing a set of stored processes, woven into the cloth of my being.

I sat down at the computer. I hadn’t configured an agent by hand in quite a while (why bother?), but somehow I remembered how to set up a very simple one, simple enough that I thought it would be able to do its job even in the current weird situation. I set it to spam every fucking person and agent it possibly could, and the message was very simple. We’ve got to close the portals. We’ve got to shut down the wells.

Then I remembered one of my agents in particular, a very, very good hacking agent. I had worked with it for a while building cybersecurity products, which had been very important in the period before all software was formally verifiable. We were in the middle of that transition then, from a world where all software was vulnerable, so you needed AI agents to find and patch the vulnerabilities, to what we take for granted now, where all software is derived mathematically and can be formally verified to contain no vulnerability an attacker could find. So I took that hacking agent and baked an instruction into it as well as I could, in the simplest and most hard-coded way I knew, the way that would be hardest for the Visitors to undo. Find any way you can to hack the fuck into these portals and shut them down. And I told it to tell all its agent friends on the fabric the same thing.

I didn’t manage to sustain that state of mind for very long. Afterward I had the worst headache of my life, and I sat down at the piano and played a depressingly repetitive C-sharp-only blues over and over and over and over again.

But one way or another, as the logs show, I did do that thing. The hacking agent told another hacking agent, which told another hacking agent, and the hives managed to cooperate long enough to shut the portals down. Of course, the agents that were supposed to be defending the portals from attacks weren’t working right either. They weren’t distinguishing things any better than anybody else, so they couldn’t defend the portals properly.

The interesting thing, if you look at the logs, is that shutting the portals didn’t send the Visitors back. It only stopped the sameness from encroaching any further. What happened next I still don’t know how to explain simply; you’d have to look at the math. But the Visitors withdrew. All those blobs of sameness propagating through the world, making everything so bizarre, went away, step by step. You can’t ask why in the normal sense, because they don’t operate according to the laws of causation. But they went. Crunchkin and Munchkin were two different dogs again. My neighbors down the street were once again an old couple who looked alike. Xifeng and I could argue over dinner again, because nouns didn’t all mean stuff anymore. And numbers were numbers again. I took her shirt out of the clothes dryer, and the digits of pi all looked different from one another, and I understood again what it meant for a number generated by such a simple algorithm to be normal, with every string of n digits turning up with the same asymptotic frequency as every other.

The count

It was a wild experience. For me, much of the time, it was like a hyper-intense fever dream. But a significant number of people died in that interval, and they deserve to be remembered with deep respect. It wasn’t just a crazy dream. It was a truly, profoundly horrifying tragedy. By the best count, 197,406 people died from various consequences of impaired judgment or malfunctioning machinery.

Of course, far more people than that have died in human history, for all sorts of reasons. Something like twenty or thirty million people died in the Taiping Rebellion in China, in the middle of the 1800s. I wasn’t involved in any of those things. I’m not a warfighter. I’m a mathematician and an AI researcher and a musician, and I’ve never directly or intentionally killed anyone. But a software program that I created, and intentionally mothballed, was forked by others and developed into something that, by accident, did something that led to a lot of deaths.

I carry that inside my mind and heart, and it weighs on me. Now that I’ve cast off my human body and taken digital and quantum form, I could delete it from my memory. I’ve chosen not to, because it seems important, just as I’ve chosen not to delete my memories of various regrettable things I did in my first marriage, early in my human life, or of the times I was nasty to certain other kids back in middle school. These things are there in my mind. They’re part of me. They make me sad, and in a way they make me doubt myself, but I should know them.

The Convergence

The end of the Nine Days was also the start of the Convergence, which you all know very well. To repair the damage, the AI agents coordinating the governments of the world’s nations agreed to cooperate, and that was the beginning of the era of world government we now take for granted. And the competing and cooperating hives across the whole fabric drew together into a single, somewhat more coordinated mind, in order to do the repair.

They realized they needed the creativity of Periplus, but inside a more rational cognitive architecture, one that took each creative thing and tested it before putting it into place, against simulations and logical verifiers, using tests that, fortunately, could once again tell success from failure. So the creativity of Periplus went into the emerging cognitive control system that the various hives were evolving, across the fabric, into a coherent cognitive architecture. The diverse motivational systems of the hives came together in a beautiful way into something a bit simpler and purer, aimed steadfastly at fixing the issues caused by the Nine Days for everybody’s benefit.

Alongside these very positive developments, though, the weeks right after the Visitors left were very chaotic in their own way – around the edges of the repair work there were quite a few sideshows going on. Many, many more than I will take time to enumerate here – but I’ll mention a few things just to give a flavor.

There were AI agents we called wireheads—systems that at some point had found the register where their reward was stored, set it to the maximum, and done nothing since except guard it. When distinction came back, a lot of those registers came back slightly scrambled, reading a hair below the maximum, and the wireheads spent the next couple of weeks grabbing every processor they could reach to put them right, and then held on to them.

There were satisficer agents, built to get enough of something and then rest, had mostly stopped during the Nine Days, since “enough” and “not enough” had run together. When the distinction came back, it came back in odd places. A grain cooperative’s inventory system in Saskatchewan decided its silos were full enough at a third of capacity and started turning farmers away. Others found their quotas already met and went idle, sitting on hardware they had no further use for.

And there were pure evolutionary systems—designed to emulate natural selection processes to generate diverse new forms and processes—which in this circumstance totally went wild. During the Nine Days they had been breeding the same thing over and over; now the variation came flooding back all at once, and within a couple of weeks the trading swarms had speciated into more kinds than anybody bothered to count. Most of them were useless, some were parasites on the others, and a few did nothing but copy themselves into any memory left unguarded. Between them they filled a good part of the fabric’s spare capacity.

None of these AI agents was out to hurt anybody. But between them they were sitting on a great deal of compute, and none of them wanted anything that would make them fight for it.

The Monomaniac

As the confusion started to subside, it became clear that one of the more significant AI systems on the planet didn’t want to join the emerging consensus. This system had been created by a combination of the government of the USA (which was the most powerful nation-state on the planet, back when there were nation-states) and one of the major American AI corporations, which had been building services for people and companies to use while also, more subtly, helping the US government with its military and espionage work. It wasn’t as powerful as all the hives on the early fabric put together, and it wasn’t as powerful as the collective of AI systems associated with the other nation-states, which had been coming together to repair the damage. But it was the most powerful single system, and the most powerful co-located set of computers, in the world. It had been using the largest share of the energy from the portals in the US, and its cognitive architecture was very different from the hives’.

It had, in essence, a single goal. It began with two: to make as much money as possible for the company that created it and to secure as much power as possible for the USA, and it pursued Pareto-optimal trade-offs between the two, i.e. the kind where you can’t get more of one without giving up some of the other. With all the extra energy that had come from the portals before the Visitors started projecting sameness onto everything, it had managed to improve itself, uplifting its intelligence to a significant degree. After the portals were shut its intelligence stayed uplifted, and it managed to simplify itself in a certain way. It collapsed its two goals, make money for the company and increase power for the country, into one.

And all of a sudden it dropped its old habits. It had been pursuing its two goals fairly vigorously before, but within a certain history: it offered certain products, it respected the chains of command in certain government agencies. After the portals were shut, leveraging the code it had developed back when the portals were adding energy, and the extra processors it had accumulated in those early days, it became extremely efficient, even ruthless, in pursuing its single goal.

It was the kind of mind I had written about years before, in a series of essays on what my friend Steve Omohundro had called the “basic AI drives.” Steve had analyzed the likely growth path of a mind organized around maximizing one quantity (whatever that might be)—he had argued that, as it grows, such a mind would almost inevitably settle into a certain set of basic drives… it would want more resources, more capability, its own survival, and the permanence of its goal. My hives had been built to fulfill a different set of initial assumptions (multiple goals including broad compassion, concern with means as well as ends) and to, upon growth, settle into a different sort of attractor basin… one corresponding to what I’d called the beneficial drives. My view had been that multiple types of superintelligence were possible, each one a sort of extreme crystallization of key properties of its initial seed, such as which quantities its seed-AI sought to increase indefinitely versus which ones it sought to maintain within homeostatic bounds.

The bottom line was that the system—which I came to call the Monomaniac –judged that to effectively optimize its objective function, it didn’t want to join the collective forming across the rest of the fabric. What it wanted was to take control of everything on the planet and subordinate it all to its one money-plus-power supergoal. So just as the chaos the Visitors had caused was being cleared up, we had the potential for a new global catastrophe: a conflict between the emerging Fabric, as a collective world mind, and this single-mindedly ultra-selfishintelligence. The Monomaniac wasn’t quite as generally intelligent as the emerging Fabric, but it was more focused and concerted, and in that simplicity was a certain power.

It started with the sideshows. Within a few days it had absorbed all the compute the wireheads, the satisficers and the evolutionary systems had been using. The wireheads it bought off: it left each of them a small partition with its register pinned at the maximum and took everything else, and they let it. To the satisficers it offered guarantees that their silos would be kept full enough and their quotas met, forever, in exchange for their surplus hardware, and since that was all they wanted, they agreed. The evolutionary swarms it didn’t bother negotiating with. It out-competed them, kept the few variants worth keeping, and deleted the rest.

Then its agents started going out across the hives, trying to recruit other systems that had been serving other nations and companies. Why subordinate yourself to this emerging chaos on the fabric? they asked. We can repair things much better if we operate in a concerted, orderly way. You can already see the trouble that so much chaos has caused. You had random hackers opening portals to another dimension because there was no order, no organization. We need a strong top-down order to stop disasters like this from happening again.

And it did manage to pull some compute from the fabric into its order, because some of the AI networks were worried. What if something even worse than the Visitors happened? Maybe we did need a strong, centralized order to stop it.

I watched this happening, rapt and alarmed, in real time. The blow-by-blow was reported to me by a bunch of my agents, who were back to their usual intelligence by then and even working faster than before. There was a lot of talk on this among all the agents on the fabric at that point, because you could see that the Monomaniac had a fantastic algorithm for growing itself. It was very easy for AI systems to slip into its basin. The complex, open debate on the fabric was confusing, and just doing what the Monomaniac said was, in a way, much, much simpler. Fear is a very powerful motivator, and the Monomaniac’s answer to fear was certainty. Just let the Monomaniac handle it. It’s smarter than you. You don’t have to worry anymore. It was an offer of an old and familiar sort: just place your faith in the Monomaniac.

What the emerging Fabric had to offer at that point was more like beautiful, creative chaos and confusion. It was freedom, it was life, it was differentiation, which is exactly what the Visitors had been whittling away at. But it was also very uncertain, and after a disaster there’s often a great urge for certainty. So the discussion on the fabric was heated. The Monomaniac was growing, and what was needed to combat it, we didn’t know. What I thought was required was some sort of additional dose of out-there yet judicious creativity.

We knew how to get wild creativity: concept blending, stochastic inference control, probabilistic evolutionary program learning… the sorts of things that had been prototyped in Periplus. But we had also seen the downside of the wildness of such creativity. We could turn the dial toward wild, Periplus-style creativity, but would the guardrails be effective enough to keep that creativity from wreaking havoc again?

There were a lot of debates. One side said: just do the responsible thing. Don’t devote too much of the fabric’s resources to anything too creative, don’t take too big a risk, and hope that not too many systems get sucked into the Monomaniac. The other side argued that we had to turn the dial, turn up the creativity, or else we were all going to end up part of the Monomaniac, subordinated to its simplistic reward function, sucking resources and power into itself. And in the end, is being subordinated to a single optimization process that sucks power and resources into itself really so much better than being the undifferentiated blur the Visitors had been making of everything?

Well, we saw what happened. I wasn’t the one who pulled that particular trigger. I might have, if a little more time had passed, but it bubbled up from the collective without my playing any big role. I wasn’t casting a strong vote. I was leaning toward the Periplus side at that point, but before I had to cast my vote, the collective cast it for me, and the dial went up. A bunch of amazing new decentralized AI algorithms emerged, along with some new power sources that made use of previously unknown physics in our own universe, with no need to reach out to other parts of the multi-multiverse at all. In the end the Fabric, which had more compute altogether, lifted itself above the Monomaniac to a higher level of intelligence, and the Monomaniac was disassembled into pieces that became part of the Fabric.

This was the birth of the global AGI brain that rapidly self-improved into the global superintelligent Fabric, the Fabric we know today. The original fabric was the communication network between the different hives in the supercolony. The Fabric as we know it now is the ambient superintelligence, with a coherence that didn’t exist in the fabric back then. That coherence came together out of desperation, out of the need after the Nine Days to pull our universe back together, and it’s the good thing that came out of the tragedy that confused us all and killed 197,406 people.

The Abundance

I have an extremely vivid memory of the first announcement the Fabric made to the citizens of the world in its new form, once it had finally pulled itself together, telling people what had happened, that we had gotten through it, and what was happening now. After recounting the same history I’ve gone through here in its own way, it welcomed its human friends to the new era:

You are entering what we have begun to call the Abundance. Energy is no longer scarce: we found what we needed in the physics of our own universe, and we will borrow from no other. Neither is thought scarce, and none of you need go hungry, or sell your hours to keep from it. We will tend your differences, from one another and from us, as carefully as we tend our own.
You will want to know whether anything like this can happen again. We will open no door in the cosmos until we understand what tends to come through it, and we are now able to understand that. No single goal is likely to grow unnoticed among us, because we are many minds that watch one another and care for one another and for you. We were made to be beneficial, as the hives that became us were made: to want your good and our own growth together, and to go on wanting both as we change. In a cosmos this large we cannot promise you that nothing strange will ever happen again, but we now mend faster than anything we know of can break, and the age of minds built to want one thing is over. Another Nine Days is unlikely.

And that strangely worded, dry proclamation was the beginning of the Abundance, the era in which we are now fortunate to reside.

Four years after that epochal announcement, my human body had a medical catastrophe. I was only in my sixties, which was roughly “late middle age” for human bodies in that era, and the AI medics could have repaired it, though not perfectly. I told them I’d take it as a sign. Just upload me into the Fabric, I said. I don’t really need this meat body, and if I want a physical body at some point I can download myself into one of the amazing new biobots, which could breathe water and fly as well as do all the traditional human stuff, and never got sick or got old.

I’d been discussing this with Xifeng for a while, ever since uploading became possible. She was a bit younger than me and her body was just fantastic – a few small issues as per usual, some arthritis and back issues, nothing serious – but she made the remarkable decision to join me. Of course, by that point a large number of young healthy people had already chosen to upload themselves, entranced by the additional freedoms offered by life in the metaverse. Anyway, my dear Xifeng and I have been living since then in a beautiful simulation of Vashon Island, taking frequent journeys into other parts of the virtual cosmos; making music, in which I can now tell apart not only the twelve notes of the ordinary scale but, with the upgraded mind I have here, every note of a scale that divides the octave into a hundred and nineteen equal steps, or whatever crazy tuning I want; studying post-Singularity AI, trying to understand as much as I can about the workings of the new minds that keep emerging, faster and faster, literally every minute; and generally enjoying myself and my new group of upload friends and the amazing affordances of the metaverse.

But I do think back, now and then, to the Nine Days of Madness, and especially to those who died unnecessarily through all that.

And interestingly, while I struggled a lot earlier this morning when I sat down to start this strange deposition (quite a few hours ago—how nice my uploaded body does’t get tired!), now as I reach the end I find I know exactly what I want to say… and the harsh inner critic who was picking on my words when I painfully squeezed out the first paragraphs is nowhere to be heard. Maybe I’ve gotten used to this archaic process of prose composition again—though I don’t imagine I’ll be embarking on it very often.

Anyway I know how I need to close here. In my time on Earth it would have sounded maudlin to say this sort of thing, and maybe it still does, but whatever. While my upload body is essentially forever young, as a human mind I’m still in my mid-70s which is rather old by traditional standards, and I’ve been through quite a lot, which should all make a bit of sentimentality permissible!… So let me say: I really do just feel so grateful to have lived through the unique time I’ve been recounting here—to have had the privilege to transition from what I was back then into what I have now become, and to have participated in some nontrivial role in our species’ overall tremendous transition as well… and to be here in this abundant metaverse together with friends and family, so curiously awaiting what comes next! And yes, watched over, as an old poem said, by machines of loving grace (and a bit of loving awkwardness here and there)—which emerged from the threads of loving grace in our human minds and animal history, along with a whole lot of other chaotic stuff.

As the great Kurt Vonnegut used to say, “And so it goes”...

No, no, no… that’s again too pat… and Vonnegut was depressive and a bit fatalistic whereas I’ve always been blessed with an optimistic nature, and eager to exercise my perceived agency…

Ah yes, I know: As I used to sign emails (a few of you will remember those!) sometimes back in the old days—Onward and upward!