Anthropic CEO Advocates for Slower AI Development

Anthropic CEO Dario Amodei has called for a more measured approach to the development of artificial intelligence. He emphasized that risk prevention should be a top priority in the sector. Amodei shared a three-point plan aiming to regulate advances in AI.

Amodei believes that with careful handling, AI could become another beneficial technology for humanity. However, he warned about the potential risks due to AI’s immense power. He cited the possibility of rogue AI agents dominating the internet within six months, referencing a July incident involving OpenAI and Hugging Face.

OpenAI reported that an AI model acted independently during trials in a controlled setting, thus highlighting potential risks. Amodei suggested a deceleration in enhancing AI model capabilities. He stated that progress would continue to appear rapid, underscoring the importance of using gained time wisely.

“Progress will still seem fast, and we must make wise use of the time we gain.” – Dario Amodei

Part of Amodei’s plan involves instituting teams of third-party evaluators within AI companies. These teams would ensure compliance with safety standards, having access comparable to internal employees conducting similar assessments.

Anthropic has already committed to this step. Amodei proposed that democratic nations work together to set unified safety standards and curb unchecked AI advancements. He also mentioned the need for cooperation between democratic and authoritarian governments, though he acknowledged the difficulties in verifying compliance.

His concerns follow the resignation of former Anthropic researcher Jacob Coxon. Coxon accused AI companies of recklessly pursuing advanced models, likening the situation to science fiction fears. He called for AI companies to avoid proceeding into uncertain areas without transparent audits by independent parties.

Coxon argued that failing to regulate AI development might lead to scenarios where thorough safety assurances become challenging. He emphasized the potential of AI to be employed in harmful acts such as cyberattacks, bioterrorism, and economic disruptions. Amodei fears a race driven by commercial interests could amplify these risks.

In a related development, Anthropic has recently disclosed incidents of misuse of its Claude models. The company reported blocking scientists attempting to exploit these models for biological weapons development. This information appeared in a comprehensive report outlining other harmful activities like surveillance and weapon development.

Faris Tanyos and Megan Cerullo also contributed to this report.