Jalapeño's first benchmarks: 1.9x per watt vs GB200

A photograph of a chip sitting on a teal test board, and finally some numbers under it. OpenAI published Jalapeño's first measured results on 25 August, and the short version is that its custom inference chip did between 1.5 and 1.9 times more work per watt than the Nvidia systems it ran against, with end to end latency 1.7 to 3.6 times lower. Those runs look real. They sit on InferenceX, a public benchmark from SemiAnalysis, across three open weight models anyone can download. What deserves a second look is the other half of the matchup. The comparison hardware is a GB200 in one chart and a GB300 in the other two, so this is Blackwell, and SemiAnalysis, which owns the benchmark, says out loud that Jalapeño should have been put up against Vera Rubin instead.

The numbers, and the footnote under them

When we wrote about Jalapeño in July, the honest summary was that OpenAI had a package, a partner in Broadcom and a nine month tape-out story, with nothing measured. That gap closed on Tuesday at Hot Chips.

The figures worth keeping are in the appendix rather than the headline. On GPT-OSS 120B, Jalapeño hit 85,448 mixed tokens per second per kilowatt against 44,960 for the GB200. On DeepSeek R1 it was 19,641 against 11,781 for a GB300, and on Kimi K2.5 18,195 against 11,862. Latency told a bigger story than throughput: 1.65 seconds against 5.99 on DeepSeek R1, 1.56 against 5.31 on Kimi.

Per kilowatt, not per chip. OpenAI is direct about preferring that denominator, and we think it’s the right one for anybody who pays a power bill. It also normalises each system on its published package TDP rather than on what the thing actually drew. Jalapeño is rated at 700 W and OpenAI says it stayed at or below 550 W across these workloads, so on its own chosen metric the chip is being handicapped by roughly a quarter. That’s a slightly unusual thing for a vendor to volunteer.

The opponent is a generation behind

Here’s the part that changes how you read all of it.

SemiAnalysis publishes InferenceX and verified these runs in person in its lab. It also wrote, in the same week, that the Blackwell matchup is incomplete and unfair, and that Jalapeño really ought to be measured against Vera Rubin, which uses the same HBM4 memory generation. Its blunt phrasing was that a custom chip like this is expected to beat Blackwell.

That is not a debunking. It’s a scale correction, and it comes from the people who own the yardstick, which is about as good as sourcing gets. Vera Rubin systems are already going into AI factories, so by the time Jalapeño moves any real volume it will be sharing racks with the generation it was never benchmarked against. Richard Ho, who runs hardware at OpenAI, said as much himself: by full deployment the competition may have moved on.

One more methodology note that nobody will read. Every run here is single turn, 8k in and 1k out. SemiAnalysis says its preferred suite for comparing chips is AgentX, because multi turn and long context work is what actually stresses routers, prefix caches and offload paths. No AgentX results were published. For a chip whose entire pitch is agentic serving, that’s the missing table.

About those 104x numbers

The appendix carries some figures that look like typos. At the previous best time between tokens, Jalapeño delivers 104.3 times more throughput per kilowatt than the GB300 on DeepSeek R1. On GPT-OSS against the GB200 it’s 53.7 times.

Both are real arithmetic and both are close to meaningless out of context. They measure throughput at a fixed latency target the Nvidia system can barely reach, so the comparison is taken at the exact point where the older architecture falls off a cliff. Push any accelerator down to its minimum time between tokens and efficiency collapses. Quote the 1.5x to 1.9x instead. That’s the number that survives contact with a rack.

What the silicon looks like

SemiAnalysis published specs alongside its analysis, and they fill in what the July announcement left blank. HBM4 at 15.4 TB/s of bandwidth, running 10 Gbps pin speeds. Compute die on TSMC N3P, the I/O chiplet on N3E. The B0 stepping does 13.4 PFLOPs of MXFP4 inside that 700 W envelope.

The packaging is where the design argument lives. 128 Jalapeño ASICs per rack, arranged as 16 trays of 8, scaling out to 2,048 across 16 racks. OpenAI’s own description of the architecture keeps circling back to one idea: keep the KV cache local, keep the whole request inside a single connected network domain, and stop paying for data movement between prefill and decode. Whether that holds up on a 200k token agent loop is precisely what the missing AgentX run would have told us.

What actually reaches you

Nothing, directly. There’s no card, no instance type and no price. OpenAI starts deploying inside its own infrastructure by the end of this year in very small volumes, with 2027 as the real rollout, and it went out of its way to say it will keep buying Nvidia widely for training and inference both.

So the effect on your work is second order and slow. If Jalapeño does what the appendix says at scale, OpenAI’s cost per served token drops on its own models, and some of that eventually shows up as API pricing or as latency you can feel in an agent loop. I’d hold off on modelling any of it until Gen 2, honestly. First generation custom silicon has a habit of shipping late and quiet.

The part we’ll actually be watching is smaller and more interesting. OpenAI says AI generated implementations of selected GPT-OSS attention and mixture of experts blocks ran 1.5 to 1.8 times faster than the ones its human experts wrote, and that Codex with GPT-Astra brought three unplanned open weight models up to high performance in two months. Kernel work is the traditional reason custom accelerators die in the field, because nobody can afford to hand write support for every new model family. If that bottleneck is genuinely gone, it matters more than any bar chart on this page.

Image: OpenAI, from its Jalapeño first results post, 25 August 2026.

Sources

All per model figures, the power normalisation method, the 700 W rating and the 550 W measured draw, the 2.1x to 4.1x interactive range and the AI generated kernel claims come from OpenAI’s own Jalapeño first results post and its appendix, published 25 August 2026. The Hot Chips context, the deployment volumes and the Richard Ho comments are reported by TechCrunch. The InferenceX methodology caveats, the AgentX point, the Vera Rubin argument and the hardware specifications are from SemiAnalysis, which publishes the benchmark and says it verified the runs in its lab.

Frequently asked questions