Kakao Releases Test Results on Optimal VM Density for VMware Infrastructure

Kakao system engineers conducted tests to derive the optimal VM density from a CPU perspective to improve operational efficiency of VMware infrastructure. To find the maximum number of VMs executable on a single PM, benchmarks were performed while varying VM CPU usage and Flavors.

Test Method and Environment

The tests targeted PMs equipped with Intel Xeon Silver 4410Y and 4214 CPUs, running CoreMark and stress-ng while increasing the number of VMs from 1 to 60. VM Flavors were categorized into three types: 2 vCPU, 4 vCPU, and 8 vCPU. Each VM's CPU usage was limited in 10% increments from 10% to 100% to simulate actual production environments.

Analysis of Performance Degradation Causes

The phenomenon of decreasing VM CPU performance as the number of VMs increases was analyzed in two main stages. In the initial stage, performance degrades as Instructions(/cycle) decrease due to an increase in Cache Misses(%). In the subsequent stage, where the PM's CPU resources become saturated, performance degradation accelerates as CPU Cycles(/sec) themselves decrease. Notably, significant performance drops occurred with fewer VMs when the VM Flavor was high-spec or CPU usage was high.

Derivation of Optimal Density and Implications

Based on the test results, the maximum number of VMs acceptable per PM was presented according to allowable VM CPU performance degradation ranges (10%, 20%, 30%). For example, on a Cascade CPU PM, when the average CPU usage of 4 vCPU VMs is 40%, it is recommended to operate approximately 21 VMs per PM to keep performance degradation within 20%. However, since these results consider only CPU resources, a comprehensive design considering memory and disk specifications is also necessary.