AI Briefing
KOSign in

PyTorch 2.15 Delivers 5-6.5x Mac GPU MoE Decoding Improvement and FP8 Support

·2026.10.22 12:30

Key point

PyTorch 2.15 improves Mac GPU MoE decoding by 5-6.5x and supports FP8.

1 / 4

Details

  • MPS backend performance for Apple Silicon Macs is significantly improved in PyTorch 2.15
  • MoE decoding speed on M3 Ultra is 5-6.5x faster compared to 2.14 (Qwen3.6-35B-A3B at 61.3 tokens/s, LFM2.5-8B-A1B at 123.6 tokens/s)
  • The main drivers of the performance improvement are native grouped matmul with a decoding-specific path, reduced GPU-CPU synchronization, and faster top-k routing
  • Support for torch.float8_e4m3fn tensors and F.scaled_mm enables up to 50% memory savings (speed is slower compared to BF16)
  • Driver memory peak during Wan2.2-TI2V-5B VAE decoding decreases from 270.135 GiB to 8.108 GiB
  • Additional performance improvements are expected in the 2.16 preview through the transition to attention Metal kernels and the removal of MPSGraph
pytorchapple-siliconmpsmoefp8gpu-optimization
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

HuggingFace BlogPyTorch 2.15 Adds FP8 Support and Boosts LLM Decoding Speed on Apple SiliconGeekNewsUnderdog Releases 'Husky', a Model-Specific Inference Engine Up to 4.5x Faster than MLXHacker NewsrGPU: Open-Source PyTorch Device for Remote GPU ExecutionPyTorchKRTorchTPU and TorchTitan Achieve 50%+ MFU on Llama 3 405B Training on TPU v7xRedditllama.cpp MoE Expert LRU Cache Boosts Qwen3.8-Flash-Next Inference to 68 tok/s on RTX 5090HuggingFace BlogTorchNeuron Profiling on AWS Trainium: System-Level Analysis and Fusion Benefits