Kakao Open-Sources 'MRTE' for Reproducing MySQL Traffic

Kakao has released MRTE (MySQL Real Traffic Emulator), an open-source tool that collects real traffic from production MySQL servers and reproduces it in test environments. The tool consists of MRTE-Collector, which captures network traffic, and MRTE-Player, which replays it on test servers, with the two modules designed to communicate via RabbitMQ.

MRTE-Collector is written in Go and compiled into native code, using the pcap library to capture network packets. Although it must run on the same equipment as the MySQL server, it maintains lightweight performance, keeping CPU usage at 2-3% and consuming only 10-15MB of physical memory while handling 30,000 QPS. In the event of Message Queue or Collector processing delays, it drops packets without blocking the user network, minimizing the impact on service performance.

The current version does not support Server-side prepared statements. This is because there is no way to externally extract the Hash ID of PreparedStatements when using the Binary protocol. However, Kakao explains that most MySQL environments use the default Client-side prepared statements or standard Statements, so there are no significant issues for production deployment.

Kakao confirmed that in tests using four servers, traffic of 30,000 QPS (including 22,000 SELECTs, 5,000 UPDATEs, etc.) was stably reproduced for three weeks, and no issues occurred during periodic restart tests of the Collector. The source code is available on GitHub.