TikTok, YouTube rack up views of AI-generated Pope sermons

AI-generated videos and audios of Pope Leo XIV are populating rapidly online, racking up views as platforms struggle to police them.

An AFP investigation identified dozens of YouTube and TikTok pages that have been churning out AI-generated messages delivered in the pope's voice or otherwise attributed to him since he took charge of the Catholic Church last month.

The hundreds of fabricated sermons and speeches, in English and Spanish, underscore how easily hoaxes created using artificial intelligence can elude detection and dupe viewers.

"There's natural interest in what the new pope has to say, and people don't yet know his stance and style," said University of Washington professor emeritus Oren Etzioni, founder of TrueMedia.org, a nonprofit focused on fighting deepfakes. "A perfect opportunity to sow mischief with AI-generated misinformation."

After AFP presented YouTube with 26 channels posting predominantly AI-generated pope content, the platform terminated 16 of them for violating its policies against spam, deceptive practices and scams, and another for violating YouTube's terms of service.

"We terminated several channels flagged to us by AFP for violating our Spam policies and Terms of Service," spokesperson Jack Malon said.

TikTok similarly removed 11 accounts that AFP pointed out – with over 1.3 million combined followers – citing the platform's policies against impersonation, harmful misinformation and misleading AI-generated content of public figures.

With names such as "Pope Leo XIV Vision," the social media pages portrayed the pontiff supposedly offering a flurry of warnings and lessons he never preached.

But disclaimers annotating their use of AI were often hard to find – and sometimes nonexistent.

On YouTube, a label demarcating "altered or synthetic content" is required for material that makes someone appear to say something they did not. But such disclosures only show up toward the bottom of each video's click-to-open description.

A YouTube spokesperson said the company has since applied a more prominent label to some videos on the channels flagged by AFP that were not found to have violated the platform's guidelines.

TikTok also requires creators to label posts sharing realistic AI-generated content, though several pope-centric videos went unmarked.

A TikTok spokesperson said the company proactively removes policy-violating content and uses verified badges to signal authentic accounts.

Brian Patrick Green, director of technology ethics at Santa Clara University, said the moderation difficulties are the result of rapid AI developments inspiring "chaotic uses of the technology."

The AI-generated sermons not only "corrode the pope's moral authority" and "make whatever he actually says less believable," Green said, but could be harnessed "to build up trust around your channel before having the pope say something outrageous or politically expedient." afp