Netflix Replaces Synchronous Registry Writes with Apache Flink for Live Ads Event Pipeline

Netflix redesigned its Ads Event Pipeline to handle live streaming spikes by replacing synchronous database writes with Apache Flink stream processing. The previous architecture relied on synchronous writes to a Metadata Registry on the ad serving critical path, creating a single point of failure and scalability bottlenecks during high-traffic events like the Jake Paul vs. Mike Tyson fight. By removing this dependency, Netflix eliminated the risk of ad serving outages caused by registry failures.

Architecture and Join Mechanism

The new system uses Kafka logs and client event streams joined in real-time via Flink's KeyedCoProcessFunction. Instead of built-in window joins, a custom operator manages a 60-minute window, ensuring that 99.999% of events are enriched immediately or within that timeframe. Events arriving later than 60 minutes are routed to a separate recovery path using Maestro and Spark batch jobs, preventing data loss for late-arriving engagement events.

Cross-Region and Deduplication Strategy

To handle network changes where client events and ad responses occur in different regions, Netflix encodes serving region information in ad tokens. This allows events to be forwarded to the correct region rather than replicating all responses globally. Deduplication was moved from an external key-value store to Flink local state with a short TTL, removing network hops and race conditions while maintaining at-least-once delivery guarantees.

Rollout and Performance Results

The transition used a shadow mode with parallel execution and automated auditing to ensure output consistency before cutover. Load tests demonstrated that a single AWS region could process hundreds of thousands of events per second during large concurrent ad breaks with negligible timeouts. The legacy Metadata Registry and associated services were decommissioned, simplifying the system and removing the critical path dependency on synchronous database writes.