Open-Source LLMs Vulnerable to Long-Reasoning Jailbreak Attacks

Testing across 10 open-source models including Phi, Mistral, DeepSeek-R1, Llama 3.2, Qwen, Gemma showed that lightweight defense techniques fail to block complex attacks.

The research team validated the following 5 lightweight inference-time defense techniques using 94 prompt injection and 73 jailbreak scenarios:

• Self-defense
• Input filtering
• System prompt defense
• Vector defense
• Voting defense

Key findings are as follows:

• While effective against simple attacks, attacks utilizing reasoning-heavy prompts consistently bypassed the defense systems.
• Model refusals and silent non-responsiveness were observed, and it should be noted that these do not simply indicate 'safety' but may represent failure modes of the model.
• This research focused on defense methods applicable to real-world local deployment environments, utilizing prompt wrappers, filters, and classifiers without retraining or costly fine-tuning.