The AI accord leaves weapons out

Past arms control succeeded only between equals, Washington now wants a two-year lead before talking to Beijing

LONDON:

Last month MI5 Director General, Sir Ken McCallum, announced the launch of the Threat Assessmnent Center (AITAC)  during his annual national security threat update at Thames House in London.

Talking about "polarising algorithms" Sir Ken McCallum pointed out how AI was supercharging the cybersecurity contest between attackers and defenders.

This warning from intelligence officials coincided with an unprecedented shift in September 2026, when heads of leading AI labs, including Anthropic, OpenAI and xAI, called for slowing the pace of frontier development. Three developments drove the shift. In one incident, AI agents hacked Hugging Face and OpenAI's systems and effectively slipped out of containment. There was evidence of attempts to use Claude for bioweapons research. And public anxiety built steadily.

Despite these warnings, everyone remains locked in a classic prisoner's dilemma. No single entity wants to slow down unilaterally, given the massive capital expenditure flowing into chip exports. The pressure is both financial and competitive, particularly as leading frontier labs head towards public offerings.

Anthropic's leaked IPO filing showed a 2025 operating loss of $8 billion, and other frontier labs are in the same position. At the same time, China's AI stocks are down 40% since June, with over $480 billion of wealth gone. Stocks are rushing to enter IPO as the STAR50 plunges into bear territory, and founders are trying to cash out. At the same time, this commercial arms race is matched at the nation-state level, where both the United States and China treat AI leadership as existential.

While Washington and Beijing dominate the frontier, the battlefield in Eastern Europe illustrates the physical reality of this race. Russia lags as much as four years behind the United States and China, and recent attacks by Ukrainian drones on three Yandex data centres powered by Nvidia A100 chips have caused considerable damage to its AI programme. The irony here is that the drones which hit the AI infrastructure were themselves AI-powered autonomous weapons.

This brings us to a bigger question. How should AI technologies be regulated in general, and autonomous weapons in particular, which now include cyber agents and kinetic drones? The White House Accord on Super Intelligence (SI), signed on September 29, fails to answer this question. It proposes internal safety controls, an oversight team and external audits that ensure continuous monitoring of models during training and deployment to prevent risks in cybersecurity, biosecurity and chemical threats.

However, it does not regulate the wider landscape of autonomous weapons or rogue AI agents capable of initiating sophisticated cyber-attacks. It does not address risks from the air, such as autonomous or AI-powered drones, whether from acts of war or from terrorism. Nor is there anything in the White House Accord on SI that legally compels signatories to take appropriate measures. Without structural changes to commercial incentives, the accord risks acting as little more than a convenient cover for labs seeking to curb loss-making capital expenditure.

Now US policy hawks are looking for a similar accord on AI with China related to AI safety and controls, perhaps modelled on Cold War arms control negotiations. At present, China's own production of AI compute is only around 4% of that of the United States, and the majority of China's computing power comes from smuggled and remotely accessed US chips. By some estimates, US AI labs are somewhere between six and twelve months ahead of the Chinese, and US policymakers believe the United States needs to extend this lead by up to two years before negotiating an AI agreement with China. They believe that closing loopholes in export controls would expand the US lead in AI, and could force China to conclude that cutting a deal soon would be preferable to an even more lopsided deal later.

This dynamic highlights a fundamental historical hurdle. Past arms control treaties succeeded because they regulated power between equals. Every past bilateral strategic arms control agreement was either negotiated between countries that were already at overall strategic parity, or agreed only after one country concluded that it was better to lock itself into a disadvantageous position because it could not win the strategic competition.

Historically, there is no example in which two states of unequal strength concluded an agreement to regulate a strategically critical technology that either caused the leader to give up its overall strategic advantage or constrained the other state's development capabilities. For example, the SALT II, START and New START (2010) agreements capped nuclear warheads for both countries at identical levels. The Anti-Ballistic Missile Treaty did the same by putting similar caps on the deployment of missile defence systems.

This does not mean an AI arms control pact is totally out of the question. Beijing is genuinely concerned about AI, and China's top spy chief has warned that it poses a serious threat to the Chinese Communist Party's rule and overall political stability. The recent market turbulence indicates that CoP-style deliberations are still possible, if both superpowers remain sincere about mitigating existential security risks and preventing an unregulated race towards autonomous warfare.

THE WRITER IS A CAMBRIDGE GRADUATE AND WORKS AS A STRATEGY CONSULTANT