Google DeepMind is worried about what happens when millions of agents start to interact

Google DeepMind, in collaboration with Schmidt Sciences, ARIA, the Cooperative AI foundation, and Google.org, has announced a $10 million funding initiative for research into the safety of multi-agent AI systems. This move comes as agent-based AI tools, capable of operating without human oversight, become more prevalent, introducing new classes of risks. The funding aims to encourage research outside of large tech companies, leveraging academia's capacity for long-term vision.

Rohin Shah, who leads Google DeepMind’s AGI safety and alignment research, highlights the urgent need for a dedicated field of study in multi-agent safety. The concern is that as more AI agents are deployed and interact, previously hypothetical risks could materialize. These risks primarily involve supercharged versions of existing internet problems, such as scams, prompt injections, and cyberattacks. Shah emphasizes the need to anticipate and address these dangers before agents are widely integrated into the economy.

James Fox of Schmidt Sciences stresses the importance of understanding the collective behavior of AI agents, which cannot be predicted by studying individual units. He points out that AI agents, especially those underpinned by large language models, may not always act rationally. Therefore, realistic simulations are crucial to observe and analyze what happens when numerous AI agents interact in complex environments.

The initiative also recognizes that some researchers hypothesize that artificial general intelligence might emerge from a "hive mind" of interacting agents rather than a single super-smart model. The growing urgency of this research is underscored by the rapid advancements in AI, with risks that were once considered hypothetical now becoming very real. This proactive funding aims to build a robust framework for understanding and mitigating the potential societal impacts of interconnected AI systems.