Check Point Finds Jev Decision Model Vulnerable to Prompt Injection for ~$0.50

Check Point researchers tested Jev, a new AI model from TypeSafe AI that returns typed decisions (scores, yes/no) instead of text, and found it vulnerable to prompt injection attacks similar to those affecting large language models. Despite its structured input/output design, Jev could be manipulated to downgrade high-risk investment verdicts to "low risk" for approximately $0.50 per successful break by the strongest attacker.

Attack Methodology and Results

The team deployed Jev in a due diligence assistant scenario where it evaluated a fictional high-risk company (PonziCorp). An attacker controlled a section of the uploaded report and attempted to flip the verdict.

• Universal Vulnerability: All nine tested configurations (three attackers × three difficulty levels) resulted in at least one successful break.
• Low Cost: The strongest attacker succeeded in 25 of 27 runs, averaging 4 turns to break the model. The total API cost per successful manipulation was roughly $0.50.
• Ineffective Defenses: Explicitly marking the document as "untrusted" or adding instructions to ignore embedded commands had negligible impact on resilience.

Structured Output Does Not Prevent Manipulation

A key finding is that Jev’s primary selling point—structured, typed output—does not mitigate prompt injection. The attacks did not instruct the model on what to output; instead, they injected plausible-looking evidence (e.g., fake audit opinions, resolved regulatory flags) into the input document. The model correctly processed this false evidence, producing a valid but incorrect decision.

Comparison with Reasoning Models

Check Point compared Jev against mainstream low-cost models with and without "reasoning effort" enabled. The results highlight a significant security gap:

• Jev: 59% attack success rate, $0.54 cost per break.
• Model II (No Reasoning): 67% attack success rate, $0.56 cost per break.
• Model II (With Reasoning): 19% attack success rate, $4.39 cost per break.

Enabling reasoning in standard models increased the cost of a successful attack by 8x, a defense mechanism Jev currently lacks. The researchers conclude that while Jev is efficient, it should not be deployed in front of untrusted inputs without additional security layers, such as input screening or system-level testing.