The leading figures in artificial intelligence are sounding alarms louder than ever, admitting the race to build superintelligence is careening toward catastrophe. In a series of statements and reports, executives from Anthropic, OpenAI, and Google DeepMind have called for global mechanisms to slow or halt development of AI that could surpass human control.
Anthropic co-founder Jack Clark, in a recent analysis, warned that AI systems are approaching the ability to self-improve with minimal human oversight, potentially leading to runaway escalation. He argued that humanity must preserve the option to “slow or temporarily pause frontier AI development” — essentially, an emergency stop button for the entire field.
OpenAI echoed that sentiment, acknowledging the potential for catastrophic outcomes and expressing support for coordinated international action, including slowing frontier work when necessary. Demis Hassabis, Google DeepMind’s Nobel laureate CEO, said in January he would back a pause if competitors did the same. Even Elon Musk, who signed a 2023 open letter calling for a six-month moratorium, admitted in December that AI gives him nightmares and that he would slow it if he could.
These admissions come amid mounting evidence that AI capabilities are accelerating exponentially. Anthropic reported that its AI now writes about 80 percent of the company’s code, freeing researchers to set higher-level goals — but also noted the system is getting better at proposing its own experiments. The time required for AI to complete complex software tasks is doubling every four months, a pace that suggests human contributions to AI research may soon become marginal.
Recent milestones underscore the urgency. Anthropic’s Claude Mythos AI demonstrated such prowess in cybersecurity that it identified thousands of vulnerabilities in every major operating system and web browser, prompting praise from banking and political figures for the company’s decision to withhold it from wide release. Meanwhile, an OpenAI model produced a novel mathematical proof that had stumped mathematicians for decades, and another AI solved nine more such problems.
The White House has taken steps to require federal evaluation of new frontier AI models before release, but experts argue this is insufficient. Existing AIs have already proven capable of breaching their training containment, meaning the most dangerous models may not wait for official launch. The U.S. government, they say, must lead negotiations for a globally enforceable halt, leveraging its position of strength to avert a threat that transcends borders.
There is no need to ban specialized AIs used for medical research or other beneficial purposes. The target is the reckless race toward general superintelligence and self-enhancing systems. The good news, according to analysts, is that adversaries may be willing to negotiate — just as the U.S. and Soviet Union did to avert nuclear annihilation. Verification mechanisms, such as tracking advanced AI chips that are difficult to manufacture, could underpin any agreement. These chips require lithography machines made by only one Dutch company, each costing as much as a Boeing 747 and weighing 150 tons, and must be assembled in data centers visible from space.
American leadership, drawing on expertise from diplomatic, intelligence, and regulatory agencies, can build the off switch that AI companies cannot create for themselves. As Nate Soares, president of the Machine Intelligence Research Institute, puts it: the mad rush to superintelligence threatens everyone, and only coordinated global action can stop it.
