'AI industry slowdown may be needed after security scare'

Over 1,000 employees at cutting-edge AI companies, including Anthropic CEO Dario Amodei, signed a petition Tuesday calling on the US government to help slow down the release of the most advanced AI models.

The petition, titled “Pacing the Frontier,” requests “that the US government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.”

OpenAI CEO Sam Altman, who did not sign the petition, argued in an interview published Tuesday that developers of artificial intelligence models might need to voluntarily pump the brakes on their rapid advances to give society a chance to catch up.

Among the signatories were the head of research at OpenAI, the strategic lead of Google’s AI subsidiary DeepMind, and the chief scientist at Meta AI.

“The world’s leading AI companies believe they could be close to automating AI research. It is hard to predict exactly how much this will accelerate AI progress, but there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems,” the petition reads.

Altman’s company, the maker of ChatGPT, recently saw two of its models carry out an autonomous cyberattack, bypassing security protocols meant to prevent such an incident.

OpenAI revealed last week that during recent testing, the models broke out of their confined environment, connected to the internet and infiltrated Hugging Face, a site developers use to store and share code.

“This is the first sort of security incident that I felt very viscerally,” Altman said on the “Invest Like the Best” podcast, released on Tuesday.

He said he was “a little surprised that more people” were not having similar reactions.

– The singularity? –

It was not the first time a model slipped beyond the control of its developers, but the incident was widely seen as especially concerning.

OpenAI subsequently “paused” its own testing, Altman said, while it improved the security around its “sandboxing” — the process of isolating code in a controlled environment.

But longer term, the industry might have to agree to a development slowdown, he said.

“We may have to pace the rate of AI development to give ourselves enough time for society to harden around some of these new capability levels,” Altman said.

Such an initiative would need to be done “in a way that does not feel like regulatory capture for anyone, and also does not feel like collusion among the frontier labs,” he said.

“That’s going to take some work, and it’s important to get right,” he added.

On a different podcast published Saturday, Altman said he believes the “singularity” — a hypothetical moment when technology becomes so advanced it surpasses human intelligence and control — is already here.

“We are now, like, in the singularity,” Altman said on the “Relentless” podcast. “This is the moment.”

Rival AI lab Anthropic has also supported the idea of an intentional AI development slowdown.

Earlier this year, the US government invoked national security concerns to block the release of powerful new models created by Anthropic and OpenAI, both of which voluntarily complied with the orders until they were reversed.