Jared Palmer Releases 'Kev', a Lightweight Decision-Making Model Based on Qwen3.5

Jared Palmer released Kev, a lightweight decision-making model based on Qwen3.5. The model is available in three sizes: Kev-0.8B, 4B, and 9B, and can simultaneously process yes/no (noul), multiple-choice (choice), and rating (score) questions in a single request. It runs on CUDA, ROCm, and Apple Silicon (MLX), with the 4B and 9B models capable of running on 32GB Macs.

Performance and Optimization

Kev-9B achieved an accuracy of 0.852 on the New Sources benchmark, slightly lower than the hosted Jev's 0.857. When running via MLX in an Apple M5 environment, it shows probability values similar to the PyTorch path while enabling efficient inference. However, the current model is slow on Apple Silicon and requires transformers >= 5.17.

Training and Deployment

Training uses the decision-v7 dataset and LoRA rank 16. It is recommended to load existing checkpoints for fine-tuning, and since training on Mac is slow, using Modal(H100) is recommended.

Limitations and Caveats

Kev-9B often underperforms compared to Jev in external evaluations, and the MMLU score gap (Kev 0.74 vs Jev 0.90) persists due to the knowledge gap in the base model. Option order can affect answers, and the server handles only one request at a time, so batching is not supported. Date calculation performance improves to 0.90 when setting KEV_DATE_FACTS=1.