AI-Powered Security Monitoring Automation for Detecting Login Anomalies with AWS Bedrock

To quickly identify credential stuffing and multi-account abuse, we built an AI-based automated login security monitoring system. Previously, we only received abnormal login alerts and had to manually check accounts and related information, which took a long time to grasp the context and respond.

Since login history alone made it difficult to distinguish attack types, we introduced an LLM. By feeding account information and login history together for contextual analysis, we could even catch similarities like phone number suffixes in ascending order or names differing by only a part, which helped overcome the limitations of a rule-based approach.

The architecture was kept relatively simple.

• A batch service periodically queries login history to detect abnormal patterns.
• When an anomaly is found, it additionally collects login history and related information, then requests analysis from AWS Bedrock.
• Bedrock's analysis results are sent to Slack, allowing near real-time situational awareness.

On the operations side, we used Bedrock Playground to check prompts and token usage, and used Prompt Management to store and swap prompt versions, enabling rapid improvements. We also adjusted the Bedrock Client read timeout to 1 minute to handle intermittent response delays.

Since adoption, we've been able to immediately grasp the context of abnormal login attempts, and the time needed to decide on a response has decreased. Results can now be checked right away even at night or on holidays, significantly improving operational efficiency.