Uppsala University OBM Design Reduces HBM Usage by Up to 82% in LLM Inference

·2026.11.14 20:12

Key point

Simulation of Optical Broadcast Memory shows 40-82% HBM savings across three models on 80,000 accelerators, with DeepSeek V4 Pro achieving 1.57x throughput.

Details

  • Optical Broadcast Memory (OBM) distributes fixed model weights via optical fiber from a central server, eliminating the need to store duplicate weights in each accelerator's HBM.
  • In a simulation of 80,000 accelerators, OBM reduced HBM usage by 40% for Llama 3.1 405B and 39% for Llama 4 Maverick, with throughput remaining at 0.96x and 0.97x of baseline respectively.
  • DeepSeek V4 Pro achieved the largest savings, reducing HBM by 8.44 PB (82%) and increasing cluster-wide throughput to 1.57x while lowering energy per token to 0.58x.
  • The 1.57x throughput gain for DeepSeek V4 Pro depends heavily on baseline memory capacity; increasing baseline HBM to 192 GB per accelerator reduces the throughput ratio to 0.96x.
  • The design relies on a conceptual multicore praseodymium-doped fiber amplifier (MC-PDFA) that has not yet been built, and the evaluation excludes the prefill stage and model training.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.