Hugging Face Releases 207 WebGPU Kernels, Boosting Browser AI Inference Speed by 2.57x

The Hugging Face WebAI team has released a collection of 207 WebGPU kernels and the JavaScript loader @huggingface/kernels to maximize AI inference performance in the browser. This library allows users to directly load and execute optimized kernels from the Hugging Face Hub, with each kernel provided as a complete package including an interface, WGSL shader template, and accuracy and benchmark cases.

Performance Benchmarks and Optimization

Compared to ONNX Runtime Web on an Apple M4 GPU, the results showed a geometric mean speedup of 2.57x across all operations and a median speedup of 1.90x. Specifically, the ai.onnx.Add operation was 3.52x faster, and Softmax was 2.11x faster, while certain Bilinear Einsum cases demonstrated speed improvements of over 10,000x. These improvements were achieved through fine-grained optimizations tailored to hardware characteristics such as workgroup size, memory access patterns, and vectorization.

Crowdsourced Validation via Fleet

To overcome the limitations of single-device results, the browser-based benchmarking tool Fleet was also launched. Fleet executes kernels on users' actual hardware to collect accuracy and performance data, supporting optimization decisions across diverse GPU and browser environments. Hugging Face plans to improve kernel variants based on this data and upstream them to the broader ONNX Runtime Web ecosystem.