AI BriefingThe winner flips depending on the model
Key point
In an apples-to-apples FP8 comparison, the advantage between Ironwood and B200 reversed depending on the model.
Details
Google's Ironwood inference benchmark and Nvidia's B200 InferenceMAX figures were compared under matched conditions.
- Compared models: Qwen3-32B dense and Qwen3-Coder-480B-A35B MoE
- Matched conditions: FP8 e4m3 weights, activations, and KV cache, same sequence length, concurrency, number of prompts, and seed
- Settings:
randomdataset,--random-range-ratio 0.8,--num-prompts 320,--max-concurrency 64,--seed 100,--ignore-eos - Stack: SGLang 0.5.10 on B200, vLLM-on-TPU on Ironwood
- Scale: 32B at 1 chip vs 1 GPU, 480B at 4 chips vs 4 GPUs
The results diverged depending on the model. For 32B dense, Ironwood had the edge, while for 480B MoE, B200 led by about 80% in the 8k/1k range in particular.
The interpretation is that dense models rely heavily on single matmul operations, where Ironwood's TPU kernels were strong, while MoE hinges on expert routing and dispatch, where SGLang's MoE handling and NVLink all-to-all appear to be better optimized.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Step 5 Preview Released: 600B Parameter MoE Model Highlights Agentic Tasks and Financial Analysis Performance