Jeff-Code Reduces Qwen3.8-27B Coding Time by 32%

Jeff-Code is a fork of the Pi coding agent, where the 0.8B Jeff model determines on each turn whether to execute the information gathering step and whether to enable Qwen's thinking mode, reducing Qwen3.8-27B's task time by 32%.

Structure and Operating Principle

• Jeff Model: A System 1 model fine-tuned from Qwen3.5-0.8B, which returns only probabilities without generating sentences.
• Decision 1 (Information Gathering): The code adapter first executes steps that do not require deep thinking, such as file reading. It is executed when the top-choice probability is 0.40 or higher, with an offline accuracy of 92%.
• Decision 2 (Thinking Level): The code-router adapter determines Qwen's thinking level (off, low, medium, xhigh). Thinking is enabled only when the xhigh probability is 0.6 or higher, with an offline accuracy of 68%.

Evaluation Results

In evaluations across 6 benchmarks and 1,242 paired tasks, Jeff-Code showed the following performance compared to the baseline (Qwen3.8-27B, xhigh on all turns):

• Pass Rate: 62.4%, showing no statistically significant difference from the baseline (62.8%).
• Time per Task: 0.68x, a 32% reduction.
• Total Time Sum: 0.86x, a 14% reduction, though some tasks took longer than the baseline.

Limitations and License

• The current report is in INTERIM status, with only English tasks evaluated; behavior for Korean instructions is unverified.
• The code is under the MIT license, and the model and adapters are under the Apache 2.0 license, but license verification for the training data and the judge model (Qwen3.8-Max) is ongoing, so confirmation is required before commercial use.