Netflix Details Three Integration Approaches for Its Personalization Foundation Model

Netflix centralized member preference learning into a single Foundation Model to reduce the maintenance burden of multiple specialized models. The team identified a gap in practical guidance for integrating large transformer-based models into existing production systems and experimented with three distinct approaches: embeddings, subgraph integration, and fine-tuning.

Embeddings via Embedding Store

The first approach uses the Foundation Model to generate profile and item embeddings, which are stored in an Embedding Store. The model is pre-trained monthly and fine-tuned daily to incorporate new titles and data. To address drift caused by retraining and daily updates, Netflix employs an embedding stabilization technique to map embeddings into a consistent space.

• Pros: Low cost, high leverage, and a low barrier to entry for application teams.
• Cons: Potential staleness due to the time gap between embedding generation and inference, limiting real-time adaptability.

Foundation Model as a Subgraph

This method embeds the Foundation Model’s subgraph directly into the downstream application model. The subgraph processes raw user interaction sequences, allowing the application to leverage deeper layers and avoid staleness issues.

• Pros: No staleness, access to deeper model layers, and potential for higher performance.
• Cons: Increased complexity in feature generation, larger model size, and higher compute costs. Netflix optimizes this by running the subgraph once per profile per request.

Fine-Tuning the Foundation Model

Similar to domain-specific fine-tuning in LLMs, application teams can fine-tune the Foundation Model on product-specific data. Teams can choose full parameter fine-tuning or freeze specific layers. This approach allows for custom output heads and objectives tailored to specific surfaces like "Trending now."

• Pros: Optimized for specific use cases and provides a strong baseline for new applications.
• Cons: Increases the number of models and pipelines to maintain, requiring careful optimization of latency and SLAs.

Netflix continues to refine these patterns, investing in near-real-time embedding frameworks and distilled versions of the Foundation Model to reduce latency.