Netflix Evolves TimeSeries Cold Storage from Kafka/Flink to Native Cassandra S3 Reads

Netflix has transitioned its TimeSeries Abstraction platform from a Kafka/Flink-based streaming solution to a native Cassandra cold tier that reads directly from S3 backups. This shift addresses the high cost of maintaining massive hot storage clusters for infrequently accessed historical data, which now spans multiple petabytes.

From Streaming to Native Backup Reads

The initial solution used Apache Kafka and Flink to stream data into S3, optimizing it into Parquet files via a daily compaction job. While effective, this approach incurred significant data transfer costs and operational overhead. The new architecture leverages existing Cassandra disaster recovery backups in S3, eliminating the need for separate data pipelines. By reading directly from these backups, Netflix reduced attached storage usage on affected clusters from 80% to 7%.

Custom Memtable and Compaction Strategy

To enable direct S3 reads, Netflix implemented a custom Backup Memtable in Cassandra. This component downloads SSTable index and metadata files from S3 to local disk, allowing efficient query planning without fetching the large Data.db files immediately. The system uses a new compaction strategy called ROCS (Read-Optimized Cold Storage) to align SSTables with token ranges, reducing read fan-out and improving tail latency. Initialization of the Backup Memtable is automated via the TimeSeries control plane, which verifies compaction completion and backup integrity using Cassandra Virtual Tables.

Performance and Isolation

The new cold tier serves thousands of reads per second with p90 latency improved by ~30% compared to the previous implementation, while keeping p99 latency under 500 ms. To prevent cold read spikes from impacting hot path performance, Netflix can deploy an isolated Cassandra cluster for cold reads. This cluster matches the original topology but runs on cheaper, storage-light instances, further optimizing costs while maintaining resilience through chunk caching and parallelized S3 access.