Kakao DKOS Removes Kubespray and Introduces In-House Provisioning, Cutting Cluster Build Time by 70%

Kakao's internal KaaS platform DKOS has removed Kubespray, which it had used for four years, and transitioned to a self-developed provisioning method without external dependencies. The previous approach involved significant operational complexity and security constraints, such as analyzing hundreds of Ansible tasks and allowing SSH access between nodes.

The new provisioning consists of three stages: Infrastructure, K8S Core, and Add-ons. In the Infrastructure stage, Packer is used to build the DKOS Base Image, which includes OS configuration and the container runtime. In the K8S Core stage, cloud-init is used to generate TLS certificates and execute deployment scripts for core components (kube-apiserver, etcd, etc.) during VM boot. In the Add-ons stage, a self-developed Helm-based installer was created to improve version management and configuration tracking.

These changes have dramatically improved cluster build speed. For a control plane of 5 nodes, generation time decreased by 70% from 700 seconds to 200 seconds, and data plane addition time was reduced by 95% from 180 seconds to 10 seconds. Additionally, managing all logic with in-house code has made debugging easier and improved service stability.