OpenAI's New Safety Framework Reads Like a Compliance Checklist, Not a Revolution
The company published two governance documents this week. They're fine. They're also exactly what you'd expect from a company trying to stay ahead of regulators.
Image credit: Lottie animation by Centre Robotics (LottieFiles Free, used with credit). · source
OpenAI dropped two governance documents this week, a "Frontier Governance Framework" and a playbook for third-party AI evaluations, and I've seen this movie before.
Back in the early 2000s, every tech company scrambling to comply with Sarbanes-Oxley suddenly discovered they'd always cared deeply about internal controls. The documents they produced were perfectly reasonable, technically sound, and about as surprising as finding coffee in a newsroom. OpenAI's new framework gives me the same feeling. It's not bad! It's just... compliance theater dressed up as thought leadership.
The Frontier Governance Framework is OpenAI's attempt to show regulators in the EU and California that they've got their house in order. The company outlines how it handles AI safety, security practices, and risk assessment. If you're expecting groundbreaking new commitments, well, you're going to be disappointed. What you get instead is a systematic documentation of practices OpenAI says it already follows.
The second document, their playbook for third-party evaluations, is actually more interesting to me. It lays out guidance for how outside groups should assess AI model capabilities and safeguards. The company is essentially trying to standardize how people test their systems, which (call me old-fashioned) feels a bit like a student writing the rubric for their own exam.
Related coverage
More in Policy
The same features that make prediction markets useful for forecasting also make them nearly impossible to police for information asymmetry.
James Chen · 2 days ago · 6 min
Pope Leo XIV's first encyclical isn't just religious hand-wringing about technology. It's a surprisingly sharp critique of concentrated power.
Sarah Williams · 3 days ago · 4 min
While tech leaders talk about alignment and safety, Leo XIV cuts through the noise with something we've been missing: a focus on actual humans.
Mark Kowalski · 3 days ago · 5 min
A tiny group of anonymous crypto whales now decides who wins billions in prediction market bets, and the SEC is watching closely.
To be fair, someone needs to figure out how to evaluate these systems consistently. The field is a mess right now, with different evaluators using different methodologies and getting wildly different results. OpenAI's playbook addresses validity, how to actually measure what you think you're measuring, and that's genuinely useful. But it remains unclear how much independence third-party evaluators can really have when the company being evaluated is also setting the standards.
The timing here is not subtle. The EU AI Act is rolling out in phases, California's SB 1047 debate showed that state-level regulation is coming whether the industry likes it or not, and OpenAI just went through a very public leadership crisis that raised questions about its commitment to safety. This framework reads like a direct response to all of that.
I've covered tech long enough to know that companies don't publish detailed governance frameworks because they woke up one morning feeling transparent. They publish them because regulators are circling, because investors are asking questions, because the PR calculus shifted. None of that makes the framework worthless, it just means we should read it as what it is: a positioning document.
The framework explicitly mentions alignment with "emerging EU and California regulations." That's the tell. OpenAI is trying to shape the regulatory conversation by demonstrating they're already doing what legislators might require. It's smart strategy. It's also not the same thing as being a safety leader.
The problem with AI governance frameworks is that they're only as good as their implementation, and we have basically no way to verify implementation from the outside. OpenAI can publish all the documents they want about their internal safety practices, but until we have actual independent auditing (not just evaluations guided by OpenAI's own playbook), we're taking a lot on faith.
Some things in the framework seem genuinely useful. The emphasis on pre-deployment testing, the acknowledgment that different models require different risk assessments, the commitment to ongoing monitoring after release. These are the basics, sure, but the basics matter and not every AI company is doing them consistently.
Other parts are vaguer than I'd like. The framework talks about risk practices but doesn't provide specific thresholds for when a model is too risky to deploy. It mentions security measures without detailing what happens when those measures fail. The playbook discusses evaluation validity but acknowledges that "no evaluation is perfect," which, okay, but what's the acceptable margin of error here?
Let me spend a minute on this evaluation playbook because it highlights a tension that nobody in the industry has solved.
Third-party evaluations are supposed to provide independent verification that AI systems work as claimed and don't pose unacceptable risks. Great idea! The problem is that frontier AI models are so complex, so expensive to run, and so difficult to probe systematically that truly independent evaluation is nearly impossible. Evaluators need access to the model, computational resources, and often guidance from the company itself about what to test and how.
OpenAI's playbook tries to address this by providing a shared methodology. But (and this is the part that bugs me) when the company being evaluated is also the one defining best practices for evaluation, the independence gets pretty theoretical. It's not that OpenAI is being dishonest, it's that the structure of the situation makes genuine independence really hard to achieve.
I talked to some folks in the AI safety community this week, off the record, and the consensus seems to be that OpenAI's playbook is a useful starting point but shouldn't be treated as definitive. Some argue that government-funded evaluation bodies need to develop their own methodologies. Others counter that the government doesn't have the technical capacity yet. Everyone agrees the current situation is inadequate.
Look, I've been covering tech since before most of OpenAI's workforce was born (that's not quite true but it feels true some days), and I've watched a lot of companies publish a lot of frameworks. The ones that mattered had a few things in common.
First, they included specific, measurable commitments. Not "we will assess risks" but "we will not deploy models that score above X on benchmark Y without additional review." OpenAI's framework is light on these specifics.
Second, they created genuine accountability mechanisms. Independent board oversight, third-party audits with teeth, public reporting requirements. OpenAI mentions some of this but the details are thin.
Third, they acknowledged failures. The most credible safety frameworks I've seen include sections on what went wrong in the past and what changed as a result. OpenAI's documents are forward-looking in a way that feels a bit too clean.
I'm not saying OpenAI is acting in bad faith. I genuinely don't know! The company has smart people who care about safety, and also a business model that requires shipping products quickly, and those two things create real tension. This framework might represent a serious internal commitment, or it might be a PR exercise, or (most likely) it's somewhere in between.
Here's what I think is actually happening. The AI industry is in the phase where everyone's scrambling to establish norms before regulators do it for them. OpenAI is trying to be the one setting those norms, which is both self-serving and potentially useful. If their framework becomes the template that other companies follow and that regulators reference, OpenAI gets to compete on a playing field they helped design.
Is that bad? Not necessarily. Somebody has to go first on this stuff, and OpenAI has more resources than most to think it through carefully. But we should be clear-eyed about what we're looking at. This is a company protecting its interests while also (maybe, possibly, we'll see) advancing the state of AI governance.
The documents are worth reading if you follow this space. They're competently written, they cover the right topics, they'll probably influence how other companies approach similar questions. They're also not the last word on anything, and anyone treating them as a definitive safety guarantee is being naive.
If you want to argue with me about any of this, my email's on the about page. I still check it more than Slack.
