This is a short one for a change—but it links to a whole bunch of more-expansive prior posts and documents and such for those who prefer more details…
Trying to lay it all out in one place. This is what we need to do to minimize pre-Singularity transition destructive-chaos and smooth our path toward an abundant and wonderful post-Singularity world for human, nonhuman and posthuman beings…
1) HOW BENEFICIAL AGI MAY ACTUALLY EMERGE, in 20 non-simple but feasible steps -
2) LLMs are a tool for getting to AGI, but not a complete seed!
There seems to be a (foolish) implicit near-consensus that if indeed human-level AGI is near, it’s going to come via current frontier LLM labs training bigger models and wrapping them in better harnesses. This is short-sighted and incorrect.
LLMs are suitable as components of AGI systems and tools for building AGI systems, but can’t themselves be the core of human-level AGI—they lack the needed capacity for innovation, and they lack meaningful agency and coherent understanding of self and world.
LLMs are no more inevitable to win the AGI race than mainframes were to win the computer race, or than Google was to own the online information market forever,
FORTUNATELY, THERE IS A BETTER WAY TO BUILD AGI, BOTH FAR MORE FEASIBLE AND FAR MORE LIKELY TO BE BENEFICIAL TO HUMANS AND OTHER SENTIENT BEINGS. See my talk from AGI-26, (start at 20:00).
https://www.youtube.com/live/wPwresMk5dg?
3) Build neural-symbolic-evolutionary agents, whose agentic loops involve not only LLMs but also symbolic working, medium-term and long-term memory operating with reasoning and creative evolution—Hyperon is one such design already on the way to completion: and with some real-world impact. https://magazine.mindplex.ai/post/how-omega-lost-its-claw
4) Create hives of these agents, with differentiated functions but shared (as well as individual) symbolic knowledge stores
5) Provide hives with default motivational systems that guide them toward beneficial actions, and are robust with respect to reasonable self-modifications and environmental changes. https://magazine.mindplex.ai/post/harmonizing-progress-and-safety-in-recursive-ai-self-improvement
6) Create a super-colony of these hives, with differentiated functions and with cross-hive communication and shared symbolic knowledge stores
7) Create robust metrics for agents and hives to use to evaluate their general intelligence capability across multiple domains. These don’t need to be non-nameable, merely usable by agents sincerely evaluating their own capability. https://magazine.mindplex.ai/post/seeding-rsi-toward-asi
8) Give numerous hives the task of upgrading their own intelligence leveraging the corpus of existing AGI designs and their own improvisations thereof, measuring progress according to the given AGI metrics. https://magazine.mindplex.ai/post/how-we-could-actually-get-a-near-term-beneficial-intelligence-explosion
9) Replace the fixed-weights LLMs in the agentic loops with “open weights LLMs with continual-learning-capable neural-symbolic caps”—thus obtaining real-time learning and rich neural-symbolic synergy. https://magazine.mindplex.ai/post/ness-the-neural-symbolic-sandwich
10) Supply the super-colony with a well-constructed “seed ontology” to guide its thinking, and let it collaboratively improve this ontology as it goes. https://magazine.mindplex.ai/post/distinction-calculus-new-math-for-minds-that-can-fork-merge-and-morph for the distinction-calculus upgrade of https://bengoertzel.substack.com/p/hyperseed-v2)
11) Implement the hives inside containers running provably secure microkernels, and then have dedicated “math hives” focus on proving the hives fulfill their specs as they evolve... while other hives focus on refining the specs and having them reviewed by the supercolony (and occasional expert humans). https://magazine.mindplex.ai/post/averting-the-cybersecurity-apocalypse
12) Complement the security of correct-by-construction with “purple team” hives of cybersecurity agents, composed of red and blue team agents leveraging neural-symbolic AI, building world-models of relevant computer systems and networks, and sharing the models and findings. https://docs.google.com/presentation/d/1gJHconEqmFFH9RE3EM9GGBmVmKkJW6V4ylkn2xte35s/edit?slide=id.p1#slide=id.p1
13) Deploy a large portion of the super-colony across a decentralized network with no central owner or controller—aking the whole thing a decentralized self-organizing organism at the hardware as well as software level—ASI:Chain anyone? NuNet anyone? https://superintelligence.io/products/asi-chain/
14) Protect the decentralized network from “copy and fork” attacks via cryptographic laterality, in which nodes exchange know-how with neighbor nodes using cryptographic secret-sharing, meaning that an attacker wanting to copy know-how out of the network would need to copy a huge percentage of the network. https://bengoertzel.substack.com/p/avoiding-agi-catastrophe-part-2
15) Roll out decentralized prediction markets and other collective decision mechanisms to allow the super-colony to form rational estimates of various aspects of its future evolution and environment -- watch out for our Fusion Social product, coming soon to an internet near u...
16) Roll out mechanisms for decentralized agent economy wherein subnetworks of the super-colony can form “shards” with their own tokenomicss driving their collective activity and development -- https://drive.google.com/file/d/1PR48QSuipECQTdlGSKo0_3nT7oIiQGvW/view?usp=drive_link or a new rough draft https://drive.google.com/file/d/1TgxASXbfL88QFjS7l_kk3nR94AbliJAc/view?usp=drive_link
17) Operate the decentralized super-colony network via decentralized collective governance that gives both humans and AGIs a say, and synthesizes ideas and inputs according to AI-powered governance regulated by an AI reputation system
18) Provide an initial ethical “constitution” defined in terms of the seed ontology, and make the ongoing evolution of this constitution one of the collective responsibilities of the super-colony—current draft Constitution here, https://docs.google.com/document/d/1c_D5ZHOxz3GW8esC3ojO2bURu5XmHCldw9DiM-vwNSk/edit?usp=drive_link
19) Provide your values and guidance and friendship to the super-colony as it rapidly evolves itself toward beneficial superintelligence—one place to get started is BGI Commons, https://bgicommons.org/
20) And before you know it—after a huge amount of work by a network of dedicated genius teams and a potentially very challenging transition period... building on the whole of human civilization and terrestrial evolution before that—the Beneficial Singularity is here !! the next phase of human and posthuman growth awakens... https://magazine.mindplex.ai/post/a-beneficial-agi-manifesto