AI BriefingSemIf: Instant Branching Decisions with a 4B Model Without Text Generation
TheoLeeCJ/SemIf
About the project
When an agent makes simple branching decisions like 'send to A if this condition is met', the process of generating and parsing sentences by an LLM is wasteful. SemIf directly reads the model's logits to return probability values instantly. The text generation loop is eliminated, significantly speeding up response times.
Based on the Qwen3.5-4B model, evaluating 21 conditions takes 5.3 seconds with the generation approach, but this tool completes it in just 1 second. It also supports a mode that loads long contexts once and evaluates multiple conditions in parallel. It runs in various environments, from CPU-only llama.cpp backends to MLX on Apple Silicon.
The interface pattern of Jev, TypeSafe's closed-source service, has been reproduced with open models. The 4B model achieved an accuracy of 84.5%, close to Jev's 88.3%, and higher performance was verified through the 27B EXL3 bridge. All benchmark data and prompt hashes are published in the repository to ensure reliability.
TheoLeeCJ/SemIf
Semantic ifs from open models, on a 3090 at home. Independent; not affiliated with Jev or TypeSafe.
Python
This introduction was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report errors, attribution issues, or removal requests via Contact.