Netflix Solves Cold-Start Problem for New Content Using CLIP Embeddings

Netflix introduced CLIP-based multimodal embeddings to solve the cold-start problem that occurs when new content is launched and to improve the accuracy of personalized recommendations. Existing models treated assets only as opaque IDs, forcing reliance on popularity-based heuristics for new content with insufficient interaction data, but the new approach immediately reflects visual and auditory features.

Visual Understanding via CLIP Embeddings

The model encodes each artwork using the CLIP image-text embedding model to generate a 768-dimensional vector, which is combined with learned ID embeddings. This approach allows the model to understand the actual content of the artwork (themes, cast, color palettes, etc.), enabling it to immediately transfer preference signals from past content with similar visual patterns, even when there is no interaction history for the asset.

Model Integration and Efficiency Improvements

Previously, separate models were trained for each canvas size (portrait, landscape, panoramic, etc.), but since CLIP embeddings are invariant to cropping or resizing, signals from multiple canvases can be pooled into a single unified model. This allows preferences for artists or genres to transfer naturally across canvases, reducing overall system complexity and maximizing personalization effectiveness.