Unsloth Releases Guide for Fine-Tuning and Local Serving of LLM-Based Decision Models

Unsloth has released a guide for fine-tuning existing LLMs (such as Qwen, Gemma, and Llama) into Decision Models by attaching small Decision Heads that return probabilities for each option instead of generating text, and for serving them locally. This follows the Jev format from TypeSafe AI and is compatible with open-weight models such as Laya from Convai Innovations and Clef from Cloudflare.

Structure and Use Cases of Decision Models

• How it works: The input state and question are concatenated into a single prompt. After the LLM reads it once, the decision head scores each option based on the last hidden state. The absence of a text generation step results in high memory efficiency.
• Question types: Supports yes/no (noul), multiple choice (choice), and scoring (score), handling up to 64 questions and 255 options.
• Use cases: Suitable for judgment tasks that require selecting one option from a fixed set, such as customer inquiry classification and tool call approval.

Training and Serving Methods

• Training: LoRA adapters and decision heads are trained together via the Unsloth Studio UI or code (FastDecisionModel). Recommended settings are 4-bit LoRA, rank 16, and a learning rate of 2e-4.
• Serving: Trained models or Laya/Clef can be exposed via Unsloth Studio's Jev-compatible Decision API for local execution. Existing Jev-integrated code can be used simply by changing the URL.

Performance and Caveats

• Performance improvements: For the Qwen3.5-0.8B model, accuracy on the typed-decisions dataset improved from 36% to 73%, and BANKING77 improved from 7% to 74%. Qwen3.5-2B recorded a validation accuracy of 81%.
• Resources: Training Qwen3.5-0.8B requires 4GB VRAM and takes 42 minutes, while fine-tuning Laya is the most efficient, requiring 2.5GB VRAM and taking 10 minutes.
• Interpretation caution: The ground truth in typed-decisions is the average judgment of teacher models, so scores indicate agreement with the teacher. Scores significantly exceeding the teacher self-agreement ceiling (0.735) suggest potential overfitting.
• License: The Unsloth decision model training code is licensed under AGPL-3.0-only, so conditions must be checked for use in internal services.