Mustafa Suleyman, co-founder of Google’s artificial intelligence research center DeepMind and current CEO of Inflection AI, has proposed a novel method for evaluating artificial intelligence. In his new book, “The Coming Wave: Technology, Power, and the Twenty-first Century’s Greatest Dilemma,” he discusses the implications of the future of AI.

Suleyman argues that the well-established Turing Test, named after the British mathematician and computer scientist Alan Turing, may no longer accurately measure AI’s cognitive abilities. The Turing Test was developed in the 1950s to determine whether a machine exhibits human-level intelligence. The test involves human evaluators attempting to distinguish between a human and a machine, with the machine passing if it can convincingly mimic human responses.

Suleyman argues that the traditional Turing Test falls short of quantifying AI’s intelligence accurately. “It doesn’t tell us anything about what the system can do or understand, anything about whether it has established complex inner monologues or can engage in planning over abstract time horizons, which is key to human intelligence,” he stated.

As an alternative, he suggests “Artificial Capable Intelligence” (ACI), which prioritizes what an AI can say and what it can do. In this new framework, a chatbot is assigned a series of short-term objectives and duties that require minimal human input. Suleyman proposes that the ultimate challenge should be the bot’s ability to convert a $100,000 seed investment into $1 million.

To achieve this, the AI bot would need to research an e-commerce business idea, devise a comprehensive business plan, locate a suitable manufacturer, and then sell the product successfully. Suleyman believes such a test would be a more accurate measurement of AI’s capabilities and intelligence, as it would go beyond basic mimicry of human conversation to demonstrate tangible, actionable intelligence. He predicts with optimism that AI will attain this milestone within two years.

This proposal follows significant advances in artificial intelligence chatbot technology. Since its release in November 2022, OpenAI’s ChatGPT has garnered attention for its ability to conduct casual conversations, generate code, and write essays. Such AI advancements have the potential to impact the global economy substantially; a recent McKinsey report estimates that the technology could contribute up to $4.4 trillion annually.

Suleyman’s argument emphasizes the need for an advanced metric to evaluate AI capabilities, reflecting the expanding scope and complexity of what AI systems are anticipated to accomplish in the real world.