Researchers in China have developed an AGI test platform for evaluating and benchmarking AGI models.

Funded by the National Key Research and Development Program of China, the Tong* test architecture consists of infrastructure, “dynamic embodied physical and social interactions” (DEPSI), and evaluation tools, the researchers report in the journal Engineering.

Reasonable, but not AGI in general

“The Tong test looks fairly reasonable,” says AGI pioneer Dr. Ben Goertzel, CEO of SingularityNet, “but it’s plainly a test of what I’d call ‘human-capable AGI,’ not AGI in general.

“This sort of test can be valuable, but I don’t think it’s really meaningful to have a single litmus test for ‘general AGI’ nor for ‘human-capable AGI.’ These are multidimensional things that can be measured in a lot of different ways. And different in-progress AGI systems are going to perform differently on different dimensions.   

Useful evaluation tools

“However, it’s useful to have some standardized environments and tests for evaluating in-progress AGI systems, as long as one interprets these as useful tools to play with, rather than as definitive measurement devices,” says Goertzel. 

“General intelligence is not something highly precisely defined, like temperature or pressure, which can be accurately measured if one just builds the right instrument (and I say this being very aware of how hard it is to build an accurate calorimeter…!).”

* “Tong” corresponds to the pronunciation of the Chinese character of “general,” as in “artificial general intelligence”

Citation: Peng, Y., Han, J., Zhang, Z., Fan, L., Liu, T., Qi, S., Feng, X., Ma, Y., Wang, Y., & Zhu, S. (2023). The Tong Test: Evaluating Artificial General Intelligence Through Dynamic Embodied Physical and Social Interactions. Engineering. https://www.sciencedirect.com/science/article/pii/S209580992300293X (open access)

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter.