AI BriefingPyTorch 2.15 released with new NCCL backend, Apple Silicon FP8 support, and CUDA 13.4 wheels
·2026.10.29 16:04
Key point
The release includes CUDA 13.4 wheels with experimental NVIDIA Rubin support and drops Python 3.10 compatibility.
Details
- PyTorch 2.15 introduces a newly written NCCL backend as the default for distributed training, replacing the legacy ProcessGroupNCCL, with rollback available via TORCH_DIST_USE_NCCL2=0 or the nccl-legacy backend.
- Apple Silicon performance improves significantly with native Metal kernels, achieving up to 21.7x faster upsample backward and 100x faster norm reductions, alongside new torch.float8_e4m3fn support.
- CUDA 13.4 GA wheels are provided for Linux x86_64, aarch64, and Windows, including experimental support for NVIDIA Rubin GPUs, while the default wheel shifts to CUDA 13.2.
- Python 3.10 support is discontinued, requiring version 3.11 or higher, and macOS 14 support is dropped for MPS acceleration, necessitating macOS 15 or later.
- FSDP2 and pipeline parallelization gain enhanced memory and communication controls, including single CUDA graph capture for gradient accumulation steps and dedicated P2P communicators for pipeline edges.
- New one-sided transfer APIs using the NIXL backend allow peer connections without process groups, enabling CUDA graph capture of transport streams.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
rGPU: Open-Source PyTorch Device for Remote GPU Execution