PyTorch ExecuTorch Adds WebGPU Backend for In-Browser GPU Inference

PyTorch models now run directly in the browser on the GPU via the new ExecuTorch WebGPU backend, eliminating the need to convert to GGUF, ONNX, or TFLite.
Benchmarks on Apple M1 Pro show WebGPU is 4–7× faster than the CPU path (Webassembly + XNNPACK) on prefill and 2–10× faster on decode for quantized Llama 3.2 1B and Qwen3-0.6B.
A single .pte file exported via torch.export runs across Chrome, Safari, and other WebGPU browsers, supporting LLMs (Llama 3.2, Qwen3, Gemma 3), vision models (SmolVLM, SAM 2), and speech models (Whisper).
All inference happens locally in the browser tab, keeping user data on the device and enabling offline use without server-side costs.

Technical Implementation

The backend uses WGSL compute shaders and a Webassembly runtime compiled via Emscripten to bridge to the browser's WebGPU JavaScript API.
It supports dynamic shapes through allocate-at-max buffers and per-dispatch workgroup recalculation, routing shapes to specialized kernels at runtime.
The system translates WGSL to native GPU languages (Metal, SPIR-V, or HLSL) depending on the platform (macOS, Linux/Android, or Windows).

The backend is currently in early development, with plans to expand operator, data type, and model coverage.
Users can test the feature in the ExecuTorch WebGPU Playground, where models run on the same WebAssembly + WebGPU runtime.
On-device training capabilities include LoRA and DiReFT fine-tuning for Llama 3.2 1B, Qwen2.5 0.5B, and Qwen3 0.6B.