Banksalad DevOps Team Reveals Infrastructure Operations and Service Stability Strategy

Infrastructure Setup and Operations

Banksalad's DevOps team manages infrastructure with an IaC (Infrastructure as Code) approach. By configuring infrastructure through code and tracking changes, they reduce mistakes through code review and can easily perform rollbacks. This enables efficient operations even with a small team.

For their main tech stack, they operate centered on AWS Cloud, and build container orchestration directly using the Kubernetes The Hard Way approach. For monitoring, they utilize Prometheus, Grafana, the TICK stack (Telegraf, InfluxDB, Chronograf, Kapacitor), and AWS's OpenSearch and Kibana. Recently, they introduced Jaeger for service tracing and Linkerd as a service mesh.

Service Stability Strategy

Given the mission-critical nature of fintech business, they focus on minimizing customer impact when incidents occur. They use 3 AWS Availability Zones (AZs) and triplicate services at minimum to increase availability. They actively leverage Kubernetes' self-healing and auto-scaling features to prevent a single failure from affecting the entire service.

Incident Response and Developer Guidance

They have a system in place to send immediate alerts when incidents occur, and operate an on-call system by introducing Opsgenie. After incidents, they analyze root causes and prevent recurrence through a blameless postmortem culture that does not blame individuals involved.

For backend developers, rather than acquiring infrastructure knowledge all at once, they recommend naturally expanding knowledge through the process of solving problems encountered in actual work such as builds, deployments, and monitoring.