Don't answer the same question twice: Interval-aware caching for Netflix-scale Druid

To solve the surge of duplicate queries created by rolling window dashboards on Druid, which handles over 10 trillion records at Netflix, the company experimentally introduced a caching layer that understands time intervals.

The core of the problem is that the same dashboard refreshes every 10 seconds, repeating nearly identical queries. For example, a popular dashboard made up of 26 charts generates 64 queries at once, and when 30 people view it simultaneously, this results in 192 queries per second.

Existing full-result cache and per-segment cache approaches didn't fit well with rolling windows whose time ranges shift slightly over time. In particular, when real-time segments are included, Druid doesn't cache results for the sake of determinism and accuracy, so repeated queries for the same data piled up as a burden on Druid as-is.

The solution is to reuse older intervals while only querying Druid anew for the most recent, unstable interval. The cache hashes the query's shape excluding the time interval using SHA-256 to use as a key, and internally uses a map-of-maps structure where time values are bucketed by 1 minute or the query granularity, whichever is larger.

The mechanism works as follows.

• Router interception: The Druid Router intercepts requests and responds first if the cache can handle them.
• Partial hit handling: If the cache can't fill the entire request, only the missing tail after the cached starting interval is narrowed down and resent to Druid.
• Merging: Cached results and new results are merged in timestamp order and returned in the same JSON shape.
• Asynchronous writes: New data received from Druid is split into buckets and asynchronously written back to the cache.

Accuracy and freshness were secured by accepting an explicit trade-off of a 5-second TTL. Furthermore, reflecting the fact that data becomes more stable as it ages, data under 2 minutes old has a minimum 5-second TTL, and after that, the TTL doubles for every additional minute of age, extending up to a maximum of 1 hour.

For sparse metrics, negative caching was also added. Buckets that are genuinely empty in the middle are stored with an empty sentinel value to prevent unnecessary re-queries, but trailing empty buckets, where data may not have arrived yet, are not cached so as not to mistakenly mask late-arriving data.

The backend uses KVDAL, with Cassandra as the storage layer. KVDAL's two-level map and per-inner-key independent TTL fit this structure well, and as a result, Netflix obtained a caching layer that significantly reduces repeated queries from rolling dashboards while still accommodating the freshness requirements of real-time analytics.