webml-kit: an ML runtime tool for browsers

webml-kit is a framework-agnostic utility that uses WebGPU and WASM to support loading and running machine learning models in the browser.

It automates the complex processes that used to be repeated in browser-based ML implementations—writing Web Worker boilerplate, wiring up postMessage, model caching, and GPU recovery—improving developer convenience.

It is built on Hugging Face's transformers.js and provides the following key features:

• Device detection and optimization: automatic detection of WebGPU, WASM, and CPU backends, and recommended dtype selection
• Efficient model management: model caching and token streaming support
• Inference management: KV-cache management and recovery when GPU is lost during inference

Users can load models and stream generated tokens in real time with just simple API calls.