Cloudflare Unveils Gen 13 Servers: 2x Edge Compute Performance via Cores Instead of Cache

Cloudflare introduced Gen 12 servers based on AMD EPYC Genoa-X two years ago, optimizing the request processing layer FL1 around a 3D V-Cache-centric architecture. However, moving to next-generation hardware, the CPUs that offered the biggest throughput gains had significantly reduced cache, and the existing software stack couldn't fully capture that benefit.

To verify this, Cloudflare collected performance counters and profiling data using AMD uProf. The results showed a significant increase in L3 cache miss rates, and data that used to stay in L3 now had to go all the way to DRAM, causing request processing time to become dominated by memory latency. An L3 hit finishes in about 50 cycles, but a miss requiring DRAM access takes 350+ cycles, making the cost of reduced cache very high.

In initial FL1 tests, Gen 13 showed increased throughput but significantly worsened latency. For example, AMD Turin 9965 saw throughput increase by up to +62%, but under high load, latency increased by more than 50%, failing to meet service quality standards. Cloudflare also tried hardware tuning, worker scaling, and CPU pinning and isolation, but none proved to be a decisive solution.

The turning point was AMD Platform Quality of Service (PQOS). Initially, allocating only a portion of L3 within a single CCD had limited effect, but dedicating an entire CCD to FL1 significantly improved throughput while keeping latency within acceptable bounds.

The fundamental solution came from a software rewrite. Cloudflare was already working on FL2, a Rust-based rebuild running on top of Pingora and Oxy, replacing 15 years of NGINX and LuaJIT code. FL2 aimed at memory safety, development speed, and overall efficiency improvements, and as a result, it ended up with an architecture that doesn't depend on a huge L3 cache.

The Gen 13 + FL2 combination proved this in real operational metrics. On Turin 9965, FL2 achieved a 50% increase in requests per CPU% compared to FL1, a 70% reduction in latency compared to Gen 12, and a 100% increase in throughput. With this combination, Cloudflare achieved up to 2x throughput, up to 50% improvement in performance/power, and 60% improvement in rack throughput, and is now rolling out Gen 13 servers in full force across the global edge.