Anthropic's Olah says AI must be guided from outside Big Tech

VATICAN CITY: Anthropic co-founder Chris Olah said on Monday that the development of artificial intelligence ​cannot be left solely to technology companies, urging greater oversight from religious leaders, ‌governments and civil society.

Speaking at the presentation of Pope Leo’s first encyclical on artificial intelligence, Olah said there was “a real possibility” that AI will displace human labor “at very large scale”.

“If that happens, supporting ​those displaced will be a moral imperative of historic proportions,” he said, sitting ​alongside the pope.

He added that companies like his operated under strong ⁠commercial, geopolitical and personal pressures that can be at odds with the broader interests ​of society.

“Every frontier AI lab … operates inside a set of incentives and constraints that can ​sometimes conflict with doing the right thing,” he said, adding that even well-intentioned researchers remain influenced by those forces.

Olah said this made outside scrutiny essential.

Anthropic is a U.S.-based company that produces the Claude ​AI tools. It has clashed with President Donald Trump’s administration, notably by insisting on guardrails ​restricting how its models can be used for military purposes such as targeting weapons autonomously or ‌domestic surveillance.

Olah ⁠welcomed the Church’s engagement with the rapidly developing technology, saying the ethical questions raised by AI extended far beyond engineering.

“The questions raised by AI are bigger than the AI research community,” he said and called for “earnest, thoughtful critics” who could challenge companies and ​help steer the creation ​of powerful new ⁠systems in a positive direction.

Olah highlighted three areas he said required urgent attention — the risk of widespread job losses, the need ​to ensure that AI benefits are extended worldwide, and the unresolved ​question of ⁠how to interpret increasingly complex and sometimes opaque system behaviour.

“AI development is concentrated in a handful of wealthy nations. How can we ensure the gains of AI are shared globally?” ⁠Olah ​said.

Monday’s event marked an unusual convergence between the technology ​sector and the Catholic Church, which has sought to position itself as a moral voice on the implications ​of rapid advances in AI.