Qwen 3.6 27B, FP8 CoT·Hermes SFT Released

This checkpoint refines Qwen 3.6 27B through 2-stage SFT and quantizes it with FP8_DYNAMIC. The goal is to combine Claude Opus 4.6-style CoT reasoning and Hermes-format tool-calling into a single model, and the final build is about 28GB, targeting single-GPU serving.

A Stage-1 BF16 merged base and a Stage-2 LoRA adapter from the same lineage were also released separately. The lineage order is Qwen3.6-27B → stage 1 LoRA → merge → stage 2 LoRA → merge → FP8_DYNAMIC quantization.

The training configuration is as follows.

• Stage 1: LoRA r=64, α=64, 2 epoch, effective batch 72, ctx 8192, final loss 0.250
• Data: nohurry/Opus-4.6-Reasoning-3000x-filtered 3,900 rows, khazarai/qwen3.6-plus-high-reasoning-500x 500 rows, Roman1111111/claude-opus-4.6-10000x 9,633 rows
• Stage 2: LoRA r=16, α=16, 1 epoch, effective batch 32, ctx 16384, final loss 0.334
• Data: DJLougen/hermes-agent-traces-filtered 3,679 rows. The original lambda/hermes-agent-reasoning-traces is based on a Kimi + GLM-5.1 setup, and about 13% was excluded during JSON validation.
• Hardware: 2x RTX PRO 6000 Blackwell (96GB each), DDP
• Software: PyTorch 2.8.0+cu128, Transformers 5.2.0, TRL 0.22.2, PEFT 0.19.1, Unsloth 2026.4.7, datasets 4.3.0
• Quantization: all Linear layers processed with FP8 (E4M3, dynamic per-token activation scale), while lm_head remains in BF16

Only one published benchmark exists, BFCL V4 live_simple, which recorded 81.40% (210/258). Evaluation was done with vLLM 0.19.1 and BFCL's QwenFCHandler, and among the 48 failures, 0 were format/JSON errors. The author is requesting independent evaluations on benchmarks such as AIME 2025, HMMT Feb 2026, MMLU-Pro, SuperGPQA, SWE-bench Verified, LiveCodeBench v6.

The limitations are also clearly stated.

• It inherits Qwen3.6's refusal patterns and tokenizer biases as-is.
• The reasoning style is heavily influenced by Claude Opus 4.6 distillation.
• Tool-calling is tuned to the <tool_call>{...}</tool_call> format, so other runtimes require an adapter.
• There is no RLHF/DPO, and the FP8 version shows a slight degradation on some edge tasks compared to BF16.
• It is explicitly stated not to be used in high-risk domains without additional evaluation.