Qwen3.5-based Small Decision Model 'Kev' Released with TypeSafe API Compatibility and Performance Metrics

Developers have released Kev, a family of small decision models based on the Qwen3.5 architecture. The project consists of 0.8B, 4B, and 9B parameter models and is distributed with pre-trained weights, training code, and evaluation data.

Architecture and Features

Kev adopts a structure combining LoRA and a pointer head with the Qwen base model. It is compatible with the TypeSafe System One API and can handle yes/no, multiple-choice, and rating questions in a single request. Input text is shared between questions, but cross-referencing is disabled to ensure independent judgments. It supports CUDA and Apple Silicon, and the 4B and 9B models can run on a Mac with 32GB RAM at bf16 precision.

Performance and Benchmarks

Model-specific performance is reported as follows:

• Kev-9B: Recorded accuracy of 0.812/0.837 and Brier score of 0.291/0.243 on New Sources dev/test.
• Kev-4B: Recorded accuracy of 0.794/0.832 and Brier score of 0.316/0.266.
• Kev-0.8B: Recorded accuracy of 0.643/0.668 and Brier score of 0.513/0.473.

The reference model Jev (New Sources dev) achieved a higher accuracy of 0.857 than Kev-9B, but direct architectural comparison is limited due to its undisclosed training data. Additionally, the rate of answering with high confidence (above 0.9) in unknowable situations was 5% for Kev-9B, lower than Jev (9%) and the previous generation Kev-8B (26%), showing improvement in reliability.

Serving and Training Characteristics

• Serving Performance: Supports CUDA and Apple Silicon. In Apple Silicon environments, latency may occur due to the absence of specific optimization kernels; Qwen3-based models are recommended for low-latency requirements.
• Training and Fine-tuning: Uses the decision-v7 dataset, and fine-tuning with self-labeled JSONL data is recommended. Training based on public checkpoints using the --init_from option can improve performance on new domains while maintaining existing performance.