KakaoTV Builds Redis Cluster Using Kubernetes Operator Pattern

The KakaoTV backend team migrated to Kubernetes to overcome the scaling limitations of their Redis cluster, which was previously operated on on-premises PM equipment. This was driven by the need to handle traffic surges during major sports events or exclusive broadcast events, where the existing approach made immediate resource expansion difficult and caused resource inefficiency during normal operations.

During the build process, they applied the Kubernetes Operator Pattern to automate cluster creation, scale-in/out, and HA handling through the definition of Custom Resources (CRD) and controller development. The controller uses reconcile logic to periodically check if the custom resource matches the actual cluster state, and automatically performs replica promotion and new pod creation in the event of a master pod failure.

Network and Performance Optimization

To prevent performance degradation of the Redis cluster and reduce development complexity, they used Host Network. The approach of accessing via Ingress required ConfigMap updates every time a pod was created and could incur performance overhead, so they adopted a method of sequentially allocating ports based on the basePort of the custom resource. This allows operating approximately 5,000 Redis pods per single Kubernetes cluster.

High Availability (HA) and Monitoring

Pod Affinity was applied to prevent single points of failure (SPOF) caused by masters and replicas being deployed on the same worker node. Additionally, they integrated Redis Exporter, Prometheus, and Grafana to collect and visualize metrics, and designed a rollback mechanism based on the Action stack to maintain consistency even in the event of controller failure.

Operational Effects and Considerations

The adoption of Kubernetes improved resource efficiency in response to traffic fluctuations and enabled scaling without precise capacity estimation. However, drawbacks were noted regarding potential recovery delays in case of controller code errors and the lack of replica read support in some Redis libraries (such as jedis). Due to the characteristic of memory usage increasing gradually, a strategy of creating a new cluster and migrating data is also employed in non-emergency situations.