AI Briefing
KO

INT3·INT2 KV Cache Released

·2026.04.22 15:54

Key point

A Metal kernel implementation combining INT3 model compression with INT2 KV cache has been released.

Details

An implementation combining an INT3 quantized model with INT2 KV cache has been released.

  • The compression loss is reported to be around +0.14 nats.
  • Custom fused Metal kernels for Mac M-series were used to jointly optimize the model and the KV cache.
  • Qwen 7B is currently available as a preview.
  • Installation is available via brew install reinforceai/spiral/spiral, and it can be run with spiral-chat.
  • The author announced plans to add GPU support later using Triton kernels, along with more efficient packing and additional model releases.
inferenceoptimizationmacosqwenquantizationllm-servingkv-cachemetal
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

RedditMica v0.1 4B Released: Open-Source Decision Model for Agent Loops Trained for Under $30RedditTextCLF Releases Calibration-Free TQ Quantization Method and Open-Source Quant FactoryPyTorchKRPrismML Releases 'Ternary Bonsai 2 27B', a Ternary Quantized Model Maintaining 98.2% of FP16 PerformanceHacker NewsApple Releases LensVLM-9B for Processing Compressed Text ImagesTLDR AIQwen Releases 'RecreationWorld' for Training AI Agents to Rebuild AppsKT CloudHierarchical Memory Architecture Needed to Resolve AI Inference Bottlenecks