From User Sequences to Scaling Laws: Multi-Stage Architecture for Meta Ad Ranking

Meta expanded its sequence learning architecture for ad recommendations. The core concept is a multi-stage sequence model that handles heavy user modeling offline and performs only lightweight ranking tasks online.

The offline user model asynchronously processes long user behavior histories to generate and cache embeddings containing deep behavioral patterns. The online ranking model combines these embeddings with real-time ad candidate signals to calculate final rankings. This allows scaling the size of Transformer models and user sequences without significantly increasing serving resource usage.

Another key technology is dense tokenization and target-aware attention. Instead of relying on manually designed sparse features, it directly learns feature interactions from data, reducing information loss and scalability limits between the sequence model and the ranking model.

This platform and related model innovations have cumulatively increased Instagram conversion rates by 6%, Facebook conversion rates by 3%, and Facebook ad clicks by 3.5%. The sequence modeling platform serves as a core component of Meta's Generative Ads Recommendation Model (GEM).