AI BriefingMica v0.5: 4B Decision Model Hits 19.8 ms Latency and Best 4B JevBench Score
·2026.10.22 04:00
Key point
The model scores 84/111 on JevBench Hard and 58.8 on JDI, outperforming larger 9B and 35B models on the latter benchmark.
Details
- Mica v0.5 is a 4B parameter decision model released under the Apache-2.0 license, designed to run on a single consumer GPU with as little as 8 GB of VRAM.
- Inference speed reaches 19.8 ms per decision at 256 tokens and approximately 1.2 seconds at 32k tokens on an RTX 6000 Ada, making it up to 3.8x faster than other 4B judges and 25 to 340x faster than API decision models.
- Accuracy benchmarks show a score of 84/111 on JevBench Hard, the highest among 4B models tested, and 58.8 on JDI, surpassing all 9B and 35B models in the comparison set.
- The model supports yes/no, multiple choice, and scoring tasks with 2 to 255 levels, maintaining 97.3% consistency when answer options are shuffled and offering a threshold setting to abstain when uncertain.
- Available formats include INT8 (fastest on NVIDIA), GGUF (BF16, Q8_0, IQ4_XS) for AMD, Apple, and CPU, with the INT8 version running 2 to 2.7x faster than Q8_0 GGUF on llama.cpp.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
llm-d 1.0 releases production-ready LLM inference stack with multi-tenant QoS and stable API
ArtCraft releases open-source IDE for AI image and video creation with 62-model catalog