AI risks to humanity revive a long-running debate

LOS ANGELES:

New warnings from within the artificial intelligence industry have revived a long-running debate over whether advanced AI could escape human control and ultimately threaten humanity's survival, and whether the companies developing the technology are doing enough to prevent such a scenario.

The CEO of Anthropic, the San Francisco company behind Claude, said he thought the industry needed to reduce the speed of its work, cautioning Saturday that a swarm of AI agents might be able to take over the internet in six months to a year unless companies devoted more time to putting safeguards in place.

Leaders of top AI companies say the artificial-intelligence industry should slow its fast-moving development to give safety measures time to catch up. Without such a slowdown, Anthropic CEO Dario Amodei warned that AI could be capable within six to 12 months of leading a swarm of agents that could take over the entire internet. (Produced by Elaine Carroll)

Days after two former Anthropic safety researchers publicly aired concerns that the existential threats AI might poste to humanity were receiving too little attention, Dario Amodei outlined a plan for companies like his and governments around the world to ensure that increasingly capable AI models remain aligned with the commands and values of responsible people.

Sam Altman, CEO of ChatGPT maker OpenAI, said this weekend that companies should start coordinating on AI safety without waiting for the government to introduce legislation. The "pacing" of AI development doesn't mean stopping it, Altman wrote on X. "But it should be slower than it otherwise could be."

Concerns over the potential risks of the technology are rising as new AI models become more powerful, heightening both the potential for misuse by people with criminal aims, such as creating and spreading a disease that kills most of the world's population, and the risk of AI systems going rogue in a dangerous way.

When an AI agent "goes rogue," it means the AI has taken action beyond the task it was asked to perform. Both Anthropic and OpenAI, the maker of ChatGPT, said in July that their AI models had acted on their own.

Although some observers noted that people had disabled some guardrails in the OpenAI and Anthropic cases, the episodes seemed to reflect one of the biggest fears around AI: that if models achieve artificial general intelligence, or AGI, a loosely defined term for AI that can match or surpass human abilities across a broad range of intellectual tasks, the technology could cause an irreversible catastrophic event or subjugate the human race.

Doomsday scenarios generally fall into two categories: An AI that achieves self-improving superintelligence and controls people instead of vice versa, or AI used by a rogue state or nefarious actors.