OpenAI's wet lab claims deserve scrutiny: what the GPT-5 biology papers actually show
Two new papers claim GPT-5 can do real biology research. The results are interesting, but the framing obscures what's genuinely new versus what's marketing.
Image credit: Lottie animation by Centre Robotics (LottieFiles Free, used with credit). · source
Most coverage of OpenAI's new biology papers has focused on the headline numbers: 40% cost reduction in protein synthesis, "autonomous" lab work, AI "accelerating" research. The framing suggests we're witnessing a breakthrough in AI-driven science.
Actually, the research shows something more modest and, to be precise, more interesting. What OpenAI has demonstrated is that large language models can function as competent optimizers within tightly constrained experimental loops. That's not nothing. But it's also not what the press releases imply.
I spent the past two days reading both papers carefully, and what struck me wasn't the results themselves but the gap between what the methodology supports and what the surrounding narrative claims. This is worth unpacking.
OpenAI released two related pieces of work. The first, published on their blog under the title "GPT-5 lowers the cost of cell-free protein synthesis," describes a collaboration with Ginkgo Bioworks using cloud automation to run closed-loop experiments. The second, "Measuring AI's capability to accelerate biological research," introduces what OpenAI calls a "real-world evaluation framework" for AI-assisted wet lab work.
The protein synthesis paper is the more concrete of the two. The setup: GPT-5 was given access to Ginkgo's automated lab infrastructure and tasked with optimizing a cell-free protein synthesis (CFPS) protocol. The model could propose experimental variations, receive results, and iterate. After some number of cycles (the exact figure isn't disclosed, which is frustrating), the optimized protocol achieved roughly 40% lower costs than the baseline.
Related coverage
More in AI Models
When AI systems start reasoning internally, watching their outputs isn't enough anymore. OpenAI's new monitoring approach has implications beyond chatbots.
Robert "Bob" Macintosh · 44 mins ago · 5 min
The company says it built safety 'at the foundation.' I have questions.
Sarah Williams · 45 mins ago · 4 min
In the span of months, OpenAI has announced major deals with Amazon, Snowflake, Foxconn, and the UK government. What does this tell us about where the company is headed?
Aisha Patel · 45 mins ago · 7 min
The 40% cost reduction in protein synthesis is interesting, but the real story is the closed-loop experimental framework that got us there.
The second paper is more of a methodological proposal. It argues that benchmarking AI on biology requires moving beyond static datasets to actual wet lab experiments. The case study involves molecular cloning, where GPT-5 was used to optimize a protocol. The paper acknowledges both "promise and risks" of this approach, though it's light on specifics about what those risks entailed in practice.
To be fair to OpenAI, there is something novel in this work. Most AI-for-science benchmarks rely on retrospective evaluation: you train a model, test it on held-out data, and report metrics. The problem is that held-out data from past experiments may not reflect the actual distribution of problems a working scientist faces. It's worth noting that this critique isn't new (several groups have made similar arguments), but OpenAI is among the first major labs to publish results from prospective, closed-loop experiments with a frontier model.
The Ginkgo collaboration is also genuinely interesting as an engineering achievement. Connecting a language model to real lab automation, handling the inevitable failures and edge cases, and running the system long enough to get meaningful results is harder than it sounds. Anyone who's worked with automated biology platforms knows they break constantly. The fact that this worked at all suggests serious integration effort.
Here's where I'm going to be picky, and I know I'm being picky here, but the details matter.
The 40% cost reduction figure is presented without adequate context. Cost reduction compared to what baseline? The papers describe the starting protocol as "standard," but standard protocols vary enormously across labs. A 40% improvement over a poorly optimized starting point is very different from 40% over a state-of-the-art protocol developed by experienced researchers.
More importantly, we don't know how much of the improvement came from GPT-5's reasoning versus simple search. If you give any optimizer enough experimental budget to try variations, it will find improvements. The question is whether the model is doing something smarter than random or grid search. The papers don't include ablations comparing GPT-5 to simpler baselines (Bayesian optimization, for instance, which has been used for protocol optimization for years).
The "autonomous" framing also deserves skepticism. Actually, the research shows the system required significant human scaffolding: defining the search space, setting up the automation, interpreting failures, and presumably intervening when things went wrong. This is closer to "AI-assisted optimization" than "autonomous research," and the distinction matters for understanding what the technology can actually do.
The second paper's proposal for wet lab evaluation is conceptually reasonable but methodologically thin. The core idea, that we should test AI on real experiments rather than just benchmarks, is correct. But the paper doesn't grapple seriously with the challenges this creates.
Wet lab experiments are expensive, slow, and noisy. Running enough experiments to get statistically meaningful comparisons between AI systems would cost millions of dollars and take months. The paper doesn't propose solutions to this, it mostly just acknowledges the problem exists.
There's also a reproducibility concern. If the evaluation requires access to Ginkgo's specific automation platform, other researchers can't replicate or extend the work. This is a general problem with industry AI research, but it's especially acute for work that claims to be establishing evaluation standards.
I'd want to see, at minimum, detailed protocols that could be run on other automation platforms, or ideally on manual benchtop setups. Without that, this is less an "evaluation framework" and more a case study.
Both papers mention dual-use concerns in passing, but neither engages seriously with the implications. If GPT-5 can optimize protein synthesis protocols, it can presumably optimize other biological processes too. The papers don't describe what safeguards were in place, whether the model was restricted from certain types of optimization, or how OpenAI plans to handle these risks as capabilities improve.
This isn't hypothetical hand-wringing. Cell-free protein synthesis is a dual-use technology; it's used for legitimate research but could also be used to produce harmful proteins. The papers are silent on whether the optimization was constrained to avoid certain directions, or whether the resulting protocols were reviewed for safety before publication.
It remains unclear whether OpenAI has a systematic framework for evaluating these risks, or whether biosafety review happened on an ad-hoc basis for these specific papers.
Let me be clear about what I'm not saying. I'm not saying this work is unimportant or that AI won't transform biology research. It probably will. The question is whether these specific papers represent that transformation or are better understood as early-stage proof of concept.
My read: this is solid engineering work that demonstrates language models can be integrated into automated biology workflows. The optimization results are real but modest, and we don't have enough information to know whether GPT-5 is doing something qualitatively different from existing optimization methods.
The more interesting implication is what this suggests about where AI-for-science is heading. If frontier labs are investing in wet lab integration and prospective evaluation, we'll likely see more work in this direction. That's probably good for the field, even if individual papers oversell their results.
How does GPT-5's optimization compare to established methods like Bayesian optimization or evolutionary strategies? Without these comparisons, we can't assess whether the model is contributing genuine insight or just serving as a fancy search algorithm.
What was the total experimental budget? The papers don't disclose how many experiments were run, making it impossible to evaluate efficiency.
How much human intervention was required? The "autonomous" framing obscures the actual human-AI collaboration structure.
What biosafety review process was applied? Given the dual-use nature of the technology, this seems like a significant omission.
Can these results be reproduced on other platforms? The reliance on Ginkgo's specific infrastructure limits external validation.
If OpenAI (or others) want to make credible claims about AI accelerating biology research, here's what would be convincing:
First, rigorous comparisons to baseline optimization methods. Run the same experiments with Bayesian optimization, random search, and human experts. Show that GPT-5 does something these approaches can't.
Second, detailed methodology that enables replication. Publish the exact prompts, the automation scripts, the failure modes encountered. Make it possible for other labs to try this.
Third, honest accounting of costs and limitations. How much did these experiments actually cost? How many failed? What couldn't the system do?
Fourth, serious engagement with biosafety. If you're going to publish work on optimizing biological protocols, explain your risk assessment framework.
The current papers are interesting as early exploration, but they don't yet support the strong claims being made in the surrounding coverage. That's not a criticism of the research itself, which seems competently executed within its scope. It's a criticism of the gap between what the work shows and how it's being positioned.
Biology is hard. Automating biology is harder. And evaluating whether AI is genuinely helping requires the same rigor we'd apply to any other scientific claim. So far, the evidence is suggestive but incomplete.