OpenAI's Custom Chip With Broadcom Is a Bigger Deal Than It Looks

Custom inference chips are different. They're optimized to run a specific model, or class of models, efficiently. That means lower latency, lower power draw, and lower cost per inference. For a robot that needs to make decisions in real time, in the physical world, those properties aren't nice-to-haves. They're table stakes.

So when OpenAI starts investing seriously in custom silicon, I read that as a sign the company is thinking seriously about deployment at scale. Not just serving ChatGPT queries from a data center, but running AI in places where you can't just throw more power at the problem.

I initially thought this announcement was mostly about cost reduction for OpenAI's existing cloud business. After thinking about it more, I think it's also a bet on the inference-at-the-edge future that robotics and embodied AI will require.

The Broadcom partnership and what it tells us

Broadcom is an interesting choice of partner here. The company has deep expertise in custom ASIC design and has worked with hyperscalers before, including Google on the TPU. They know how to take a company's specific workload requirements and turn them into silicon.

What we don't know yet is how much of the design work OpenAI contributed versus Broadcom, what the manufacturing node is, or what the production timeline looks like. Those details matter a lot for assessing whether this chip will actually be competitive. Bloomberg's reporting gives us the broad strokes but not the technical specifics.

Honestly, I'm not sure this holds up as a fully formed competitive threat to Nvidia in the near term. Custom silicon takes years to mature. Google's TPUs went through multiple generations before they became genuinely best-in-class for certain workloads. OpenAI is starting from a later position in the market, with the advantage of knowing what mistakes others made, but also with the disadvantage of not having years of silicon design experience in-house.

The more interesting question is what happens in three or four years, after OpenAI has iterated on this design a couple of times.

The vertical integration logic

There's a broader pattern here that's worth naming. The companies that tend to win in technology over long time horizons are the ones that control the full stack. Apple is the obvious example: hardware, software, and services, all designed to work together. Google has been moving in this direction with TPUs and now with its own Arm-based server chips. Amazon has Graviton and Trainium. Microsoft is reportedly working on its own AI chips too.

OpenAI has been the conspicuous exception to this pattern. It built one of the most capable AI systems in the world entirely on top of someone else's hardware. That gave it speed and flexibility in the early years, but it also meant every dollar of revenue was partially subsidizing Nvidia's margins.

The Broadcom chip is the first real step toward changing that.

For the embodied AI space specifically, this matters because the companies building humanoids and other physical AI systems are going to face the same vertical integration pressure. A robot that runs on off-the-shelf compute from a third-party vendor is always going to be at a disadvantage compared to one where the compute is co-designed with the software. Figure, Physical Intelligence, 1X, Boston Dynamics, all of them will eventually face a version of this question.

Some of them are already thinking about it. Nvidia's partnership with Figure got a lot of attention, but that's still OpenAI-on-Nvidia-hardware, just with a robotics-specific software layer. It's not the same as owning the silicon.

What OpenAI is actually building toward

This is where I want to think out loud for a minute, because I think the chip announcement connects to something bigger that OpenAI has been signaling.

The company has been increasingly public about its interest in agents, systems that don't just answer questions but take actions in the world over extended periods. The OpenAI blog recently highlighted how Omio, a travel platform, is using OpenAI to power conversational travel experiences and build toward being an AI-native company. That's a software story, but it points to the same underlying bet: that AI is moving from a tool you query to an agent that operates continuously on your behalf.

Agents that operate continuously are much more compute-intensive than chatbots that respond to individual queries. They need to maintain context, take actions, observe results, and update their plans, potentially for hours or days at a time. That's a very different inference workload than answering "what's the weather in Berlin."

Custom silicon optimized for OpenAI's specific model architectures could make continuous agent operation dramatically cheaper. And cheaper continuous operation is what makes agents viable for everyday use cases, including, eventually, physical agents running in robots.

I'm speculating here, and it's worth saying so. OpenAI hasn't publicly connected these dots in the way I'm connecting them. But the pieces fit.

The questions I can't answer yet

A few things remain unclear to me after reading through the available reporting.

First, what does this mean for OpenAI's relationship with Nvidia? The two companies are deeply intertwined. OpenAI trains its largest models on Nvidia hardware and presumably will continue to do so for the foreseeable future. Custom inference chips don't replace training chips. But the dynamic between them is going to get more complicated as OpenAI builds out its own silicon capabilities.

Second, will OpenAI use this chip exclusively for its own products, or will it eventually offer access to third parties through its API? If external developers can run models on OpenAI's custom hardware at lower cost, that changes the competitive picture for cloud AI services significantly.

Third, and this is the one I find most interesting for my beat: will OpenAI make this chip available to robotics companies building on its models? If you're building a humanoid that runs GPT-4o or whatever comes after it, having access to purpose-built inference hardware could be a meaningful advantage. The company didn't disclose anything about licensing or external availability.

Tbh, I think we're going to be waiting a while for answers to most of these.

Where I land on this

I think the Broadcom chip is genuinely significant, even if the immediate practical impact is limited. It's a statement of intent more than a finished product. OpenAI is telling the market, and telling itself, that it wants to control the hardware layer.

For the humanoids and embodied AI world, the implications are a bit downstream. The chip isn't designed for robots. It's designed for data center inference. But the logic that led OpenAI here, the need for compute that's optimized for your specific workload, is exactly the logic that robotics companies are going to have to grapple with as they scale.

The companies that figure out the full stack first, software, models, and silicon, are going to have a structural advantage that's very hard to replicate. OpenAI is clearly trying to be one of those companies.

Whether they pull it off is a different question. Custom silicon is hard. It takes time. And OpenAI is entering a market where Google, Amazon, and others have years of head start. This is based on limited public information, and the real test will come when we see actual performance data and production volumes.

But the direction is right. And in this industry, getting the direction right early matters a lot.

James Chen · 19 hours ago · 3 min