Netflix Conductor: The Next Chapter

Netflix has completed a major rebuild of its Conductor workflow orchestration engine, moving to version 4.0. The primary bottleneck in previous versions was the engine's memory model, which loaded entire workflow states into memory, limiting workflows to around 2,500 tasks. The new architecture introduces a 'Partitioned Task Model' that separates workflow metadata from task data, allowing the engine to scale to 30,000 tasks per workflow—a more than 10x increase. This rewrite also delivered a roughly 40% reduction in p99 workflow-evaluation latency in production. Conductor currently backs roughly 200,000 workflow definitions across 150 applications, executing on the order of 420 million workflows per month. The update includes asynchronous workflow evaluation to eliminate lock contention, new 'Run Strategies' for controlling concurrency (such as sequential execution per key), and managed tasks that integrate directly with Netflix's Titus (containers) and Stratum (serverless) platforms. Additionally, a new type-safe Java DSL (Workflow SDK) replaces JSON authoring, improving developer experience and enabling rapid adoption for complex workflows like Netflix Ads.