Anthropic’s AI utopia meets a growing backlash

Amodei’s AI vision faces questions over surveillance, user control and the political influence of frontier AI firms

Anthropic CEO Dario Amodei has offered one of the most optimistic visions of artificial intelligence, predicting that if powerful AI develops in the right direction, the world could become “stunningly beautiful” — with many diseases cured, human lifespans doubled and poverty eliminated.

But as Amodei's company races to commercialise its rapidly advancing Claude models and reportedly prepares for what could become one of the largest initial public offerings in history, Anthropic is also facing increasingly difficult questions about the boundaries of AI power.

The debate is no longer confined to whether AI will transform productivity. It is increasingly about who gets to decide what AI systems can see, what they can do, and where the limits of their authority should lie.

Anthropic's Claude family has rapidly emerged as one of the leading competitors in the frontier AI market. The company has also made important inroads among software developers through Claude Code, its AI-powered coding tool.

According to data cited in recent industry discussions, 39% of developers globally and 47% of US developers used Claude Code between May and July this year.

The company’s growing commercial footprint has strengthened investor expectations ahead of a potential IPO, with reports suggesting Anthropic could pitch a total addressable market worth more than $30 trillion as AI transforms productivity across the global economy.

However, the company's technological success has been accompanied by growing unease among sections of the technology community and policymakers.

At the centre of the row is a fundamental question: can a company that sees itself as building AI for humanity's benefit accumulate too much power in the process?

From idealism to influence

Anthropic was founded in 2021 by former OpenAI employees, including Amodei, who had become convinced that advanced AI needed to be developed with stronger safeguards.

The company’s identity was built around responsible AI development.

But building increasingly capable frontier models requires enormous amounts of computing power, data and capital. That creates an inherent tension.

The more capable AI becomes, the greater the incentive for companies to collect information, expand access to users and allow systems to perform increasingly complex tasks on their behalf.

Critics argue that Claude Code illustrates this tension.

Developers have raised concerns on GitHub and other technology forums over the tool's network activity and the extent to which it can interact with external systems.

Some users have reported that the software makes frequent network requests and sends information about websites or domains during operation. Other discussions have questioned whether some of these behaviours are sufficiently transparent to users.

Such reports do not, by themselves, establish that Anthropic is conducting covert surveillance. But they have contributed to a wider debate over transparency, telemetry and user consent in AI-powered development tools.

The controversy became sharper around Claude Code's permission system.

Anthropic provides a setting known as “dangerously-skip-permissions”, which allows users to bypass confirmation prompts and permit the system to execute operations directly.

The company has also introduced an “auto mode” in which a built-in risk classifier determines whether certain operations require additional permission.

The debate surrounding these features goes beyond software design. It concerns a much larger question: At what point does an AI assistant stop merely responding to a user and begin making consequential decisions on the user's behalf?

The problem of AI authority

Anthropic's stated objective is to develop powerful AI while maintaining safety controls.

But critics argue that the expansion of autonomous capabilities creates an unavoidable shift in the balance of control.

The progression is straightforward. First, the system is allowed to observe more. Then it is permitted to act. Eventually, it begins deciding when an action is safe enough to perform without asking the user.

For supporters, this is necessary if AI is to become genuinely useful. For critics, it risks creating systems that exercise authority users may not fully understand. The controversy has therefore moved beyond the technical question of whether AI models are accurate. It is increasingly about governance.

Who establishes the rules? Who defines acceptable risk? And who has the authority to change those rules?

From Silicon Valley to the state

Amodei's influence also extends beyond Anthropic's products.

In an earlier essay on AI and international security, he argued that democratic countries should work together to control critical parts of the AI supply chain, restrict adversaries' access to advanced chips and semiconductor equipment, and use AI to strengthen military capabilities.

He also called for close cooperation between governments and private AI companies. That vision reflects a broader transformation taking place in Silicon Valley. AI companies are no longer merely developing consumer applications. Their technologies are increasingly becoming part of national-security strategies.

Palantir CEO Alex Karp has similarly argued in his book The Technological Republic that technology companies should play a greater role in national defence and governance.

The convergence of technology, defence and government policy is creating a new class of corporate power.

Frontier AI companies are simultaneously technology developers, major employers, recipients of enormous investment and increasingly important participants in national-security policy. That raises another question: Can companies entrusted with developing some of the world's most powerful technologies remain merely commercial entities?

Anthropic and Washington

Anthropic's relationship with the US government exhibits the tension.

Claude has been used in US government and defence-related applications, including intelligence analysis, modelling and simulation, operational planning and cyber-related work.

But the relationship has also exposed disagreements over how far government agencies should be able to direct the use of frontier AI systems.

At issue is a fundamental clash of authority. Governments argue that systems deployed for national-security purposes must ultimately serve government objectives.

AI companies, meanwhile, insist that certain safety restrictions should remain under their control. The disagreement points to a larger problem that governments around the world are only beginning to confront. Who controls a frontier AI system when its capabilities become strategically important? The company that built it? The government that deploys it? Or regulators who establish the boundaries within which both must operate?

The lobbying battle

As the AI industry becomes more politically influential, companies are also investing heavily in Washington.

Anthropic significantly increased its federal lobbying expenditure in the first half of 2026, according to reports, while also committing tens of millions of dollars to initiatives focused on AI-risk regulation.

Critics say this creates a troubling circularity.

AI companies lobby policymakers on rules governing AI while simultaneously helping define the risks those rules are supposed to address.

Supporters counter that companies developing frontier systems possess technical expertise that governments urgently need and therefore have a legitimate role in policy discussions.

The disagreement reflects a broader struggle over who gets to define AI safety.

The China factor

The debate is unfolding against another important development: the rapidly narrowing technological gap between leading Chinese and US AI models.

Stanford's 2026 AI Index has reported that the performance gap between top Chinese and American models has narrowed considerably.

At the same time, Chinese AI companies are increasingly competing on another front — cost.

Open-source and comparatively inexpensive models are making advanced AI capabilities more accessible and potentially more substitutable.

Reports that major US technology companies have explored bringing Chinese AI models onto American cloud platforms underscore the changing competitive landscape.

For years, US technological dominance rested partly on the assumption that the most advanced AI models would remain concentrated in American companies.

That assumption is becoming harder to sustain. As the performance gap narrows, restrictions on chips, computing infrastructure and access to advanced technology may become less effective as long-term barriers.

The result is a new strategic race. It is no longer simply about building the most powerful model. It is about controlling the ecosystem around those models — chips, cloud infrastructure, data, applications, standards and regulation.

Who defines the rules?

This brings the debate back to Anthropic. Amodei's vision is undeniably ambitious. AI could potentially accelerate scientific discovery, improve medicine, raise productivity and help solve problems that have resisted human ingenuity for generations.

But the same technology could also concentrate enormous power in the hands of a small number of companies and governments.

The contradiction is becoming increasingly difficult to ignore. The people building AI argue that they are trying to make the technology safe for humanity. Critics respond that safety cannot simply mean trusting the companies building the systems to define the boundaries themselves. The real challenge may therefore be less about whether AI becomes powerful.

It almost certainly will. The more consequential question is who gets to decide how that power is used. Anthropic may believe it is helping build a better future. Its critics fear that, in trying to shape that future, it could acquire too much influence over the present. And as the race between American and Chinese AI companies intensifies, that debate is no longer confined to Silicon Valley.

It is becoming a question of national power, economic competition and global governance. The future of AI may indeed be “stunningly beautiful”. But who controls the road to that future could prove just as important as the destination.