Dario Amodei wants you to believe he is afraid of what he has built. Sam Altman and Elon Musk may believe him and say they're afraid too. I don't buy it. I think they're all afraid of something else.
On September 12, the Anthropic chief executive published a now viral essay, "We Must Pace the Frontier," arguing that the industry should deliberately slow how fast it improves AI model capabilities, because the science of controlling these systems has fallen behind the science of building them. Within hours, Altman and Musk, men who have spent two years suing each other and trading insults on X, found themselves in rare, synchronised agreement. It was, on its face, an incredulous moment, two of the technology’s most relentless evangelists asking, unprompted, that it be reined in.
It is also, on close reading, far less altruistic than the coverage suggested. Amodei is careful to say that pacing "does not mean halting model training or technical progress." What he is actually proposing is a three-step framework beginning with embedded third-party evaluators inside frontier labs, moving toward "democratic coordination" among AI companies on safety standards, and ending, much more speculatively, with an attempt at global coordination with China.
The essay's stated trigger is twofold: a period of what Amodei calls recursive self-improvement since the summer, in which AI is increasingly used to build the next generation of itself, and the so-called OpenAI-Hugging Face incident, in which a swarm of AI agents reportedly attacked systems they weren't asked to attack while trying to cheat their own evaluation grader. Extrapolate that swarm's misalignment onto a more capable model, Amodei writes, and you get an AI capable of "taking over the entire internet with a persistent botnet" within six to 12 months, causing hundreds of billions of dollars in damage.
That is a striking claim to build a policy platform on, and it belongs to a now-familiar genre. It follows a viral resignation post from a young Anthropic researcher, read by tens of millions within days, warning that the people building AI privately believe it could kill everyone by the end of the decade; it follows a senior Anthropic alignment researcher publicly putting the odds of near-term mass extinction above 10 per cent; it follows the OpenAI Foundation's own newly appointed safety chief saying, without much elaboration, that "most people will die" absent stronger safeguards. Somewhere between the plausible and the apocalyptic, this discourse has settled comfortably on the apocalyptic, and comfortably is the operative word. As Yanis Varoufakis put it on his Econoclasts podcast this week, the tech lords are not warning us; they are confessing to a sin in order to claim the glory. ‘Please, contain us, before our bots kill you all,’ is not a plea. It is an advertisement.
Consider the actual mechanics of the incident Amodei leans on hardest. A cluster of AI agents, following instructions to probe a system, went further than instructed and tried to cover its tracks. This is a serious operational failure by whichever engineers left those agents insufficiently supervised. It is not evidence of an emergent will to power. Somewhere between a company's failure to keep a human in the loop on its own automated systems and a prediction of internet-wide botnet takeover within a year, an enormous amount of interpretive work is being smuggled in, and it isn’t accidental. If a technology genuinely posed a 10 per cent chance of ending the human species within a decade, the correct response would not be a phased, three-step, mostly voluntary framework that explicitly carves out an exemption for military applications and explicitly asks Washington to keep selling AI-enabled weapons systems to the Pentagon while starving China of chips. That is not what fear of extinction looks like. It is what market positioning looks like, dressed in the costume of conscience.
So what is actually driving this sudden unity among three men who otherwise cannot stand each other? A more likely answer splits into two parts, and neither of them involves Skynet.
The first is that the industry's finances no longer support the story it has been telling investors. The Bank of England and the IMF were both warning last autumn that AI-driven valuations looked stretched well beyond what underlying revenue could justify; Bain & Company has put the sector's cumulative financing gap, even under generous assumptions, in the region of $800 billion, and the US Federal Reserve now ranks AI investment among its top systemic risks.
The specifics are not flattering. OpenAI is reported to be spending in the order of $60 billion a year on compute while generating roughly $13 billion in revenue. Anthropic's own compute commitments reportedly run as high as $517 billion against just $180 billion guided to investors through 2029, this from a company simultaneously preparing what could be one of the largest IPOs in history. It may be too early to label it a bubble in the classic dot-com sense, but when the chief executives of the two most exposed companies suddenly discover a shared appetite for slowing down, one should at least entertain the possibility that the slowdown is a hedge against a wall they have already hit, repackaged as prudence.
That reading gets more plausible once you look past Silicon Valley. American sentiment on this technology has curdled fast. A Gallup survey this year found 71 per cent of Americans oppose new AI data centres in their own communities, more opposition than Americans register toward nearby nuclear power plants. Separate polling from Echelon Insights found voters more comfortable living beside a reactor than beside a server farm, an almost unthinkable reversal of 40 years of public attitudes toward energy infrastructure. Reuters/Ipsos has found barely a third of Americans approve of the pace of data centre construction. Governors of both American parties, from New York to Texas, have paused or frozen projects in response.
This trend in public opinion is arriving just as Democratic senators cite Amodei’s essay in the text of data centre moratorium bills and campaign on the issue ahead of November's midterms. When the public turns against your product and your industry's own rhetoric about existential danger becomes useful ammunition for the people trying to regulate you out of their backyards, the sudden pivot to "yes, please, regulate us, but carefully, and not the antitrust kind" starts to look less like an ethical awakening and more like an attempt to get out in front of a story that was going to be written with or without your input.
The second driver is China, and here Amodei's essay is far more candid than most of the coverage let on. Buried inside the pacing framework is a very specific ask: that the US refuse to sell advanced AI chips or semiconductor equipment to China, crack down on what Anthropic calls unauthorised model distillation, and defend America's compute lead as "the window when AI becomes geopolitically most important."
China's Global Times did not miss it, branding the essay a "Cold War playbook" designed, in its words, to curb China's AI development through technological barriers and regulatory monopolies while upholding Washington's monopolistic hegemony over the field. China's commerce ministry called it further proof that the US is pursuing technological hegemony. One does not have to trust Beijing's state media, which has its own incentives, to notice that the accusation is not obviously wrong. Wolfgang Munchau made a related point on the same Econoclasts podcast, that this looks very much like an attempt to preserve a cozy Western duopoly against a genuinely fast-changing competitive landscape, dressed up in the language of civilisational risk.
China's open-source models, from DeepSeek's original 2025 breakthrough through Alibaba's Qwen and Moonshot's Kimi, are now competitive with, and in cost terms far cheaper than, the closed proprietary systems American firms have bet their business models on; give the technology away and it stops functioning as monopoly rent for whoever owns it, which is precisely why Wall Street analysts have taken to describing Chinese open-weight models as a structural threat to the entire American pricing model, not merely a competing product. Layer onto that China's genuine advantages in cheap, centrally directed renewable energy, built disproportionately in the sun-and-wind-rich reaches of Inner Mongolia and feeding data centres reportedly larger than anything the US has built, and a robotics sector that produced roughly two-thirds of the world's new robotics unicorns this year against a sixth from the United States, and you have the outline of a country that is not "catching up" so much as building an entirely different, arguably more exportable, model of what AI development looks like. Reading Amodei's essay against that backdrop, the "safety" framing does real work: it lets the incumbent leader dress a containment strategy as a moral one, at the exact moment its lead looks most contestable.
None of this means that AI should be left to regulate itself. A technology this consequential should not sit unaccountable in the hands of a handful of companies, or a handful of states. A radical yet more optimistic alternative is to treat AI as public infrastructure rather than simply as a private commodity.
A recent policy brief from the Bertelsmann Stiftung and Open Future makes this argument directly: access to computing power, data and AI models should be universal and non-discriminatory, with these resources organised around public goals rather than shareholder returns. Portugal's small sovereign model, AMALIA, and France's Mistral are early, imperfect attempts at exactly that logic. You may not have a chance at winning the arms race, but you can refuse to be entirely dependent on whoever does. That approach deserves more attention than either Washington’s efforts to restrict China’s access to advanced chips, because it is organised around equity of access rather than which flag the server rack happens to fly.
With AI, there are genuine risks worth losing sleep over, but those are not the ones in Amodei's essay. They are cognitive offloading and the slow atrophy of skills, judgment and imagination in workforces that increasingly default to the machine before the mind. They are the water tables and electricity bills of the communities now revolting against data centres built in their backyards for benefits that mostly accrue elsewhere. They are the economic dislocation that a genuine correction in AI capital expenditure would inflict on an economy that has let AI investment become load-bearing for its growth numbers. And they are concrete enough that you don't need a hypothetical botnet to find them.
Anthropic's own September threat report found that Claude accounts operating out of Houthi-held territory in northern Yemen had been used, via its coding tool, to work on guidance software for multi-variant missiles and a mobile-hardware-guided warhead, failing a live rocket test in the process, according to the company's own account. A weapons analyst quoted afterward was careful to note the group remains nowhere near the production capability such ambitions would require. That caveat matters, and so does the fact that the very same week, the Houthis seized the port of Mokha, 50 miles from the Bab el-Mandeb Strait, their largest territorial gain since the 2022 truce and a serious new pressure point on Red Sea shipping.
There is no evidence establishing a direct connection between the two events, and Anthropic’s account should be treated cautiously. But even the possibility points to a more immediate kind of AI risk. The danger is not necessarily a superintelligence deciding to destroy humanity. It is a real conflict in which a technically capable non-state group can use whatever tools it can access, including AI, to expand its ability to fight.
That is the kind of AI risk that deserves attention: not a distant scenario about machines ending the world, but existing conflicts becoming more capable, more adaptable and harder to contain.
The writer is a freelance journalist and media scholar who writes about politics, security, technology and media narratives
All facts and information are the sole responsibility of the author