Integrating LLM-Based AI into SOC Threat Detection and Response

SOC teams had to simultaneously solve a triple challenge: AI capability to read new attacks, on-premise security to protect sensitive logs, and manageable operating costs.

IGLOO Corporation developed the generative AI-based security assistant AiR (AI Road) to solve this problem, choosing an on-premise architecture that protects data instead of the cloud. Since real-time LLM inference had to be processed within a secure environment while also reducing the high power consumption and cost of GPUs, Rebellions' NPU and SDK were adopted as the core infrastructure.

AiR performs threat judgment, explains the reasoning behind its judgments, and conducts predictive analysis, helping SOC operators respond to complex situations faster and more accurately. Because the Rebellions SDK broadly supports classification AI, explainable AI, and generative AI models, IGLOO was able to easily migrate its existing GPU workloads to an NPU-based on-premise environment.

In prototype testing, the NPU showed higher tokens-per-second (TPS) throughput than GPGPU, while power consumption was reduced to about half. Power efficiency relative to performance was 1.6x higher compared to GPU, and TCO—including hardware purchase costs, space, and long-term power and operating costs—was also significantly lowered.

• Maintaining on-premise security: Sensitive logs and internal data are not sent to an external cloud
• High-performance LLM inference: Real-time inference performed within a secure environment
• Power and cost savings: Half the power consumption compared to GPU, lower TCO
• Deployment flexibility: Lowers the barrier to SOC adoption with a compact appliance form factor instead of large servers