W8A8 Quantization SDK 'Cider' for MLX Released

Mininglamp AI has released Cider SDK, developed to overcome the limitations of the MLX framework. While existing MLX only quantizes Weight and keeps Activation in FP16, Cider supports W8A8 (8-bit Weight, 8-bit Activation) quantization.

In tests based on a 4,516 token context on M5 Pro, Prefill speed improved by about 11%, from 2.84 seconds to 2.52 seconds, while Decode speed remained stable at about 80 tok/s.

This SDK operates by registering custom Metal kernels as MLX primitives, and provides optimal performance by utilizing INT8 TensorOps on chips M5 or higher. (On devices M4 or lower, it automatically falls back to the existing path.)

In terms of accuracy, it also showed a very low Perplexity (PPL) increase rate compared to FP16 for the Qwen3-8B and Llama3-8B models, demonstrating an efficient balance between performance and precision.