OpenAI’s Jalapeño shows strong early inference performance

OpenAI has detailed Jalapeño, a self-designed inference ASIC developed with Broadcom and built from a blank slate for LLM inference. Design work began in the middle of 2024 and reached manufacturing tape-out in ~16 months, an unusually fast cycle for a first-generation chip.

Early InferenceX results show Jalapeño ahead of Nvidia Blackwell across most performance-per-watt scenarios and competitive with Vera Rubin, even without multi-token prediction, speculative decoding, or prefill-decode disaggregation. The chip reached over 700 tokens per sec per user at concurrency 1 on DeepSeek R1, while Kimi-K2.5 and GPT-OSS ran at approximately 1,400 tok/sec/user. The benchmark runs were verified in OpenAI’s lab, but OpenAI supplied the numbers, and the full InferenceX suite and AgentX workloads have not yet been tested.

Jalapeño uses HBM4, a simplified memory system, and a homogenous architecture aimed at reducing latency and data movement across varied inference workloads. Production is scheduled to ramp over 2027, with most output expected near the end of next year. At rack scale, the design reaches 128 Jalapeño ASICs per rack and can connect up to 2,048 XPUs, while OpenAI’s next deployment target is 100MW.