Cloudflare Saves 100TB of Memory by Optimizing DNS Cache Structure

Cloudflare's DNS service platform, Big Pineapple, stores over 250 billion DNS cache entries. Even a mere 1-byte memory waste per entry resulted in costs exceeding 250GB across the entire fleet, making storage structure optimization essential.

Over 50% Reduction in Memory Usage

The cache entry storage method was improved five times, reducing memory usage per entry by over 50%. As a result, approximately 100TB of memory was freed up across the entire fleet, equivalent to the RAM capacity of 130 Gen 13 servers.

Performance Improvement Effects

The memory efficiency improvements did not lead to performance degradation. On the contrary, due to reduced allocation counts and improved memory locality, insert throughput increased by 43% and lookup latency decreased by 19%.

Key Optimization Techniques

• Changed from Vec<T> to Box<[T]>: Since DNS responses are not modified after storage, the unnecessary capacity field (8 bytes) and excessive heap space were eliminated. This saved 64 bytes per entry.
• Used a single list and offsets: Replaced the method of storing Answer, Authority, and Additional sections as separate lists with a single list and u16 offsets, saving 28 bytes per entry.
• Optimized struct padding: Integrated boolean fields into bitflags and optimized struct alignment to achieve additional memory savings.