zml/llmd releases v20261106.0, a multi-platform LLM inference server with sub-2GB images

·2026.11.07 07:10

Key point

The new release supports continuous batching, prefix-caching, and speculative decoding across NVIDIA, AMD, Intel, TPU, and Metal hardware.

zml/llmd released version llmd-20261106.0, a universal inference engine supporting NVIDIA, AMD, Intel, TPU, and Metal platforms.
Docker images are significantly smaller than alternatives, ranging from 142 MB for Metal to 272 MB for TPU and 1.96 GB for CUDA.
The server features continuous batching, prefix-caching, and speculative decoding (DFlash, DFlash2, DSpark) with automatic tensor and expert parallel sharding.
Supported models include the Qwen 2, 3, 3.5, 3.6, and 3.8 series, Gemma 3 and 4, LFM2.5, Mistral 3, Ministral, Mistral Large 4, LLaMa 2 and 3, DeepSeek 4.1 Flash, and GLM 5.2 and 5.3.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.