Kakao Reveals Method for Redistributing MySQL Shard Data Using ADT

Kakao has released an example of using its proprietary data transfer tool, ADT (Async Data Transfer), to redistribute data from a MySQL master DB across multiple shards. To address the limitations of traditional Range or Modulus-based sharding and the need for Scale-in/out in cloud environments, it implemented two handlers (BinlogHandler, TableCrawlHandler) that process Binlog changes and existing data simultaneously. To prevent data conflicts, it uses REPLACE statements during Binlog processing and INSERT IGNORE statements during crawling, and proposes various execution strategies, including lock usage during sequential or concurrent processing and deletion event history management. Additionally, it establishes full scans as the principle for data integrity and introduces two verification methods depending on whether service downtime is acceptable. However, it explicitly notes that the data verification part is still at the theoretical verification stage internally at Kakao.