Kakao Reveals Architecture and Operational Technologies of Real-Time Recommendation System 'Rubix'

Kakao's real-time recommendation system 'Rubix' features a structure that reflects user feedback within seconds and applies it to content rankings. It uses Apache Kafka for message queues and Apache Spark Streaming for data stream processing, adopting Spark considering the development team's proficiency in the Scala language. In terms of response speed, it processes 99% of requests within 3ms and 99.9999% of requests within 150ms. For scalability, it is designed with a Stateless structure and utilizes Couchbase, a distributed NoSQL database, handling 200,000 QPS during peak times as of April 2016. For fault tolerance, it implements automatic recovery features using Apache Mesos, Marathon, and Aurora, blocking fault propagation between microservices and applying fallback policies. As of April 2016, it has recorded 99.998% availability since service launch, maintaining zero downtime since September 2015.