Scaling LLM Post-Training at Netflix

While pre-training broadens language ability and world knowledge, post-training is the stage that aligns models with real service requirements. To apply LLMs to recommendations, personalization, and search, Netflix built an internal Post-Training Framework that reflects domain data and multi-turn interactions.

At the experimental level, it may seem like all you need is to prepare data and train a Hugging Face model, but at production scale, problems surface across four axes: data, model, compute, and workflow. In particular, the key challenges are loss masking for precisely controlling which tokens are included in the training loss, sequence packing to reduce wasted batch padding, sharded loading under FSDP, and the explosive memory growth of logits with large vocabularies.

To abstract away these problems, the framework provides the following:

• Data: dataset abstractions for SFT, reward modeling, and RL, cloud/disk streaming, and asynchronous on-the-fly packing
• Model: support for modern architectures such as Qwen3, Gemma3, Qwen3 MoE, and GPT-OSS, built-in LoRA, and a high-level sharding API
• Compute: job submission that scales from a single node to hundreds of GPUs, MFU monitoring, and checkpointing that covers parameters, optimizer, dataloader, and data mixer
• Workflow: a hybrid single-controller + SPMD execution model that supports everything from SFT to online RL, where rollout, reward scoring, and policy update are intermixed

The underlying stack layers PyTorch, Ray, and vLLM on top of Netflix's internal GPU platform, Mako. Developers work primarily with configuration files that select recipes, allowing research teams to focus on model design and experimentation rather than infrastructure complexity.

The key design shift is scaling from SFT to RL. SFT can be handled with an SPMD model where each worker runs the same step function in parallel, but on-policy RL requires rollout generation, reference model inference, reward scoring, and policy optimization, each demanding different roles and resource coordination. To support this, the driver is no longer a thin launcher but has become an active controller that manages the timing of each stage and the allocation of cluster resources.

In this process, Netflix prioritized flexibility that goes beyond a simple fine-tuning tool, capable of handling architectural variations, non-standard vocabularies, and domain-specific sequences. In other words, this is a framework whose goal is not just to "run training" but to "reliably operate complex post-training."