AI BriefingKimi K2.6, 32x MI50 9.7tok/s
·2026.05.01 07:24
Key point
On 2 nodes with 32 AMD MI50s, Kimi K2.6 int4 achieved 9.7 tok/s output and 263 tok/s input.
Details
Running moonshotai/Kimi-K2.6 int4 across 2 nodes with 32 AMD MI50 32GB GPUs achieved 9.7 tok/s at 136 output tokens and 263 tok/s at 14,564 input tokens.
- Inference stack: vllm-gfx906-mobydick
- Setup: 16 GPUs x 2 nodes, connected via 10G Ethernet
- Power consumption: about 640W idle, about 4,800W peak during inference
- Related code:
ai-infos/vllm-gfx906-mobydick - Server script: a modified version of
openai_server_kimi.py, based on the official vLLM example
