Jevper Released: A Jev Classification Wrapper for OpenAI-Compatible Models with Automatic Logprobs Detection

A lightweight wrapper library Jevper for performing Jev classification on OpenAI-compatible models has been released. This tool independently implements TypeSafe AI's System One wire format, designed to be compatible with various backends such as llama.cpp without requiring the TypeSafe API or SDK.

Automatic Logprobs Detection and Optimization

The core change is that method="auto" is now the default setting. Jevper observes the provider's logprobs support in real time to automatically determine the optimal classification method.

• Determination Criteria: It detects 4xx errors requiring the logprobs field, responses missing logprobs, and responses containing only sampled tokens to determine lack of support. 5xx errors are considered temporary issues and are not cached.
• Caching and Cost: Determination results are cached on the client per model and interface, incurring an additional cost of 1 call (2 calls for reasoning models) on the first invocation.
• Limitations: Logprobs or grammar-based classification supports up to 26 options for a single token, while structured JSON can handle up to 255 options. If more than 26 options are provided, it responds with JSON without logprobs.

Reasoning Model Support and Error Handling

For reasoning models, when configured with ReasoningConfig(effort="medium"), native reasoning is applied on the Responses interface, while a two-step 'think-then-classify' approach is automatically applied on Chat Completions. The reasoning process is stored in response.reasoning.

• Error Recovery: If reading logprobs fails (in auto mode), it automatically switches to structured JSON. For malformed answers (MalformedAnswer), one correction retry is performed.
• Retry Policy: Exponential backoff is applied for HTTP 429, 500, 502, 503, 504, 529, and connection errors.
• Dependencies: The only runtime dependency is pydantic; the openai library is used for testing only. It is distributed under the Apache-2.0 license.