The Pope's AI letter is about power, not technology

Because this framing matters more than any specific AI regulation.

Look, we've spent years in this industry debating technical questions. Alignment. Interpretability. Safety benchmarks. These are real issues and smart people are working on them. But the pope's letter, whether you're Catholic or not (I'm not, for the record), cuts to something the technical debates often miss.

The question isn't just whether AI systems work correctly. It's who decides what "correctly" means. It's who benefits when autonomous systems replace human judgment, and who bears the cost when those systems fail.

I covered the early days of self-driving cars, and the pattern was identical. Companies talked about safety statistics and sensor capabilities while quietly assuming they'd get to define the acceptable tradeoffs. How many pedestrian deaths are tolerable for reduced traffic fatalities? Who decides? The engineers? The shareholders? The people who can't afford to live in neighborhoods with good public transit?

These aren't technical questions. They're political ones, and the tech industry has historically been allergic to admitting that.

Is this just another institution trying to stay relevant?

Maybe! I don't know the Vatican's internal motivations and I'm not going to pretend I do.

But relevance-seeking doesn't make the argument wrong. The Verge reported that reactions from the tech industry ranged widely, with nearly everyone believing the document would be influential (the article cut off before I could see the full picture, which is frustrating but that's modern journalism for you).

What I find interesting is the Anthropic partnership. Here's a company that positions itself as the responsible AI lab, the one that takes safety seriously, standing next to the pope. That's not an accident. It's positioning.

And positioning isn't inherently bad! But it does suggest that at least some AI companies see value in aligning themselves with institutional critiques of the industry. Whether that's genuine concern or sophisticated PR, well, remains unclear. Probably both.

What happens now?

Probably nothing immediate. Encyclicals aren't legislation. The pope can't fine OpenAI or shut down data centers.

But institutions shape discourse, and discourse shapes what becomes politically possible. The Catholic Church has 1.4 billion members globally. That's a lot of voters, a lot of consumers, a lot of people who might start asking uncomfortable questions about the AI systems being deployed in their hospitals, their schools, their workplaces.

I've watched the autonomous vehicle industry go from "we'll have full self-driving by 2020" to "well, actually, geofenced robotaxis in specific cities with perfect weather" over the course of a decade. The technical challenges were real, sure. But so was the political backlash when it became clear that these companies were testing experimental vehicles on public roads without, you know, asking the public.

AI is following a similar trajectory. The young founders (and they're always young, or at least they seem young to me at this point) talk about transforming everything, democratizing intelligence, solving problems that have plagued humanity for millennia. And some of that might even happen! But the gap between the vision and the reality is where the politics lives.

The pope's letter is an intervention in that gap. It's saying: before we argue about whether AI can do X or Y, let's talk about who gets to decide what X and Y should be.

So what should we actually do?

I don't know. But what do I know, I'm just a guy who's been watching tech cycles for thirty years.

What I will say is this: the robotics and AI industries would benefit from taking institutional criticism seriously rather than dismissing it. Not because the pope is necessarily right about everything, but because the concerns he's articulating are shared by a lot of people who don't have encyclicals to publish.

Workers worried about automation. Communities worried about surveillance. Parents worried about what algorithms are showing their kids. These aren't fringe concerns, they're mainstream, and they're going to shape the regulatory environment whether the industry engages with them or not.

The smart play, it seems to me, is to engage. To actually grapple with the power questions rather than retreating into technical jargon. To admit that deploying AI systems at scale is a political act, not just an engineering challenge.

Will that happen? History suggests probably not, or at least not until there's a crisis that forces the issue. The self-driving car industry didn't get serious about safety communication until after the fatalities made headlines. Social media didn't get serious about content moderation until after the congressional hearings.

Maybe AI will be different. Maybe the presence of an Anthropic executive at a papal announcement signals something genuinely new, a willingness to engage with institutional critique before the crisis rather than after.

Or maybe it's just good PR.

Either way, the pope's letter is worth reading. Not because it tells you anything new about how AI works, but because it articulates, pretty clearly, why a lot of people are worried about how AI is being deployed. And those worries aren't going away just because the technology keeps advancing.

If you want to argue about any of this, my email's on the about page. I still check it, unlike some people.

Sarah Williams · 20 Jun · 4 min