An Interview with OpenAI's Richard Ho on Why Jalapeño Is a Groundbreaking AI Chip

OpenAI's Jalapeño AI inference chip, announced at Hot Chips in late August 2026, demonstrated superior performance per watt compared to Nvidia Blackwell on open-source models like GPT-OSS, DeepSeek R1, and Kimi K2.5 according to SemiAnalysis InferenceX benchmarks, though rankings may change with the arrival of Vera Rubin systems.
The architecture employs a first-principles approach for LLM serving, minimizing data movement by directly connecting 64 dedicated cores to HBM banks. This design removes the need for shared memory coherence management found in traditional GPUs and CPUs, improving efficiency at the cost of a more complex programming model.
AI agents accelerated the design process, enabling a 9-month tapeout by parallelizing tens of thousands of Power, Performance, and Area (PPA) optimization runs and discovering kernel optimizations beyond human heuristic limits.
The team, comprising approximately 100 engineers, prioritizes fungibility over disaggregated architectures to maintain data center flexibility, and plans for Gen 2 and 3 chips with a deployment strategy requiring step-function improvements in metrics like intelligence per watt rather than minor incremental gains.
Richard Ho, OpenAI's Hardware VP, previously worked on Google's first-generation TPU and Lightmatter's photonics, and now leads the GPT-Infra effort to extend Jalapeño's lessons to broader semiconductor and infrastructure cost reduction.

Key point