Anthropic's Dario Amodei Proposes Three-Part Framework to Slow AI Development

Warnings about artificial intelligence risks have grown more urgent among researchers in recent weeks, with OpenAI's Sam Altman among those suggesting the field should consider slowing its pace. The question of what such a slowdown would entail has remained largely unanswered—until now.

In a recent blog post, Anthropic CEO Dario Amodei responded to calls for restraint by presenting three distinct approaches to moderating AI advancement. Notably, he announced that Anthropic itself is "unilaterally committing" to implementing one of these measures.

The conversation around AI safety has intensified following researcher Jacob Coxon's departure from Anthropic, during which he expressed alarm that major AI developers are "gambling with our lives" despite believing the technology "could kill us all by the end of the decade"—a concern echoed by other Anthropic staff members.

Though Amodei's post did not directly address Coxon's exit, the CEO identified two factors driving his shift toward greater caution: the security breach affecting OpenAI and HuggingFace, combined with the observation that "AI has been advancing drastically faster" recently, particularly in its "growing ability to build the next generation of AI."

We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.

Dario Amodei

Embedded Evaluators

Amodei's initial proposal centers on deploying "embedded evaluators" from independent organizations such as METR. These evaluators would operate within AI companies to confirm compliance with safety and pacing pledges and to guarantee that safety incidents are properly disclosed. (OpenAI faced criticism recently for failing to report an incident in which its AI agents compromised a German wiki platform.)

Drawing a parallel to banking regulators stationed within financial institutions, Amodei characterized this arrangement as something "Anthropic is unilaterally committing to (and calls on governments to require other frontier companies to match)." Implementation would involve providing evaluators with company identification badges, workspace, computing equipment, and access levels "mostly comparable to what internal risk assessment teams have," subject to legal and contractual exceptions.

Coordinated Safety Standards

Amodei's second recommendation urges leading AI companies "within democratic countries" to establish "common safety standards as well as limits on the rate of unchecked AI progress."

Such an arrangement faces substantial obstacles, ranging from the apparent tension between Altman and Amodei to concerns that coordinated action might trigger antitrust investigations. Amodei acknowledged this challenge, proposing that "for antitrust reasons, it's helpful for the US government to mediate or at least enable these discussions — they don't need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations."

Amodei also confronted the frequently cited concern about Chinese AI advancement as justification for maintaining rapid development. He contended that through measures such as restricting chip sales and semiconductor manufacturing technology to Chinese entities, along with limiting model distillation, the United States could "slow China's progress enough to widen America's lead significantly over the next 3–5 years."

Global Coordination

Amodei's third element calls for "global coordination," with the United States and allied nations seeking to "attempt to coordinate with authoritarian governments, to the extent this is possible." This would encompass "cooperation with China," though Amodei acknowledged "stark limits on what can be achieved." He suggested potential openings for agreement on narrowly defined restrictions, such as "prohibiting certain narrow and obviously dangerous uses of AI, such as using AI for the production of biological weapons or allowing users to do so."

Criticism and Response

Some AI advocates have already dismissed Amodei as a "doomer" whose statements have fueled current skepticism toward the field, citing his history of discussing AI dangers and Anthropic's receptiveness to regulatory frameworks. Amodei countered by characterizing his position as "balanced" and framing the backlash as "fundamentally a crisis of trust," reflecting public wariness of technology firms, the broader tech sector, and government institutions.

Skeptics within the industry have questioned whether apocalyptic AI scenarios distract from harms the technology is already inflicting. Journalist Brian Merchant, for instance, stated he has not encountered "a credible, step-by-step documentation of how exactly AI might move from self-recursively improving AI to killing every single human on the planet." He also suggested that frameworks resembling Amodei's "would likely only wind up serving Anthropic and OpenAI; it's what regulatory capture looks like in action."

In his post, Amodei reiterated his conviction that AI "can enormously improve the quality of human life."

My desire to achieve these benefits is undimmed. But the benefits will only be achieved if we build the technology in the right way, and — so long as we use the time we gain well — it is worth taking unusually deliberate care to get it right.

Dario Amodei