A startup called World Labs has raised $230 million to build Artificial Intelligence (AI) technology that can understand and operate in the three-dimensional physical world, Reuters reported.

The startup is the brainchild of Fei-Fei Li, a heavyweight in the AI sector who is often called the “godmother of AI.” Among other achievements, Li developed ImageNet, a large data set that has been instrumental to waves of breakthroughs in computer vision, in deep learning, and AI at large.

World Labs will strive to build “Large World Models (LWMs) to perceive, generate, and interact with the 3D world,” notes the startup’s announcement. “We aim to lift AI models from the 2D plane of pixels to full 3D worlds – both virtual and real – endowing them with spatial intelligence as rich as our own.”

Li’s 2023 autobiography titled “The Worlds I See: Curiosity, Exploration, and Discovery at the Dawn of AI” tells the story of ImageNet and other recent AI milestones. At the heart of AI, explains Li, is “the power of data at large scales.” Big Data is, indeed, at the heart of both Large Language Models (LLMs) and LWMs.

The funding round was co-led by Andreessen Horowitz. In a commentary, Andreessen Horowitz partners Martin Casado and Sarah Wang note that, while Large Language Models (LLMs) have had a huge impact over the last few years, language is just one way we humans reason and communicate. The physical world is spatial, and we first understand and reason about it by seeing and interacting with it. The envisioned short-term applications of LWM technology include computer games and Augmented/Virtual Reality (AR/VR). Longer term applications could include AI-powered autonomous robots, a critical technology that is likely to be first prototyped with software robots acting in VR worlds.

In an episode of the Andreessen Horowitz podcast titled “The Future of AI is Here” – Fei-Fei Li Unveils the Next Frontier of AI,” Casado interviews Li and World Labs co-founder Justin Johnson on the rapid ongoing expansion of AI technology and the prospect of LWM-powered machines.

Grokking the world

The prospect of LWM technology promises machines that understand the physical world intuitively, from inside – or, using the delicious term coined by Robert Heinlein, machines able to “grok” the world and act autonomously in the world.

LLM technology has enabled spectacular advances in understanding and generating human language, but this is not likely enough for Artificial General Intelligence (AGI). Recent developments with enhanced reasoning ability such as OpenAI’s Strawberry promise multimodal AIs with a wider range of abilities. LWMs with human-like spatial intelligence that groks the physical world like we do could soon add more and very important capabilities to the AI toolbox and advance toward AGI.

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter.