CyberSecQwen-4B: Why Cybersecurity Defense Needs Small, Specialized Local Models

Frontier models excel at general-purpose tasks, but they have several critical limitations in the cybersecurity defense environment. Representative issues include data leakage risk when sensitive data is sent to external APIs, API cost burden from processing large-scale alerts, and the impossibility of execution in air-gapped environments.

CyberSecQwen-4B is a specialized 4B-scale model designed to solve these problems. Rather than simply reducing the size, it is optimized for specific tasks such as CWE classification, CVE-to-CWE mapping, and structured CTI (Cyber Threat Intelligence) Q&A.

The CTI-Bench results compared against Cisco's Foundation-Sec-Instruct-8B model are as follows:

• CTI-MCQ: CyberSecQwen-4B scored 0.5868, showing 8.7%p higher performance than the 8B model.
• CTI-RCM: It maintained 97.3% of the 8B model's performance.
• Efficiency: Despite having roughly half the parameter count, it proved comparable or superior performance on practical security tasks.

The entire pipeline for this model was performed on a single AMD Instinct MI300X instance. Using 192GB of HBM3 memory and the ROCm 7 environment, efficient training was completed in bf16 precision without quantization or model sharding.