Meta Andromeda Achieves 1 Billion Ads Per Second Throughput via Model-System Co-Design

• Meta Andromeda now scores every eligible ad at over 1 billion ads per second on a single GPU server, a 100x throughput lift compared to the 2024 system.
• A new Retrieval Mixture-of-Experts (MoE) architecture achieves near-ranking-level model performance using approximately 300x fewer inference-time FLOPs than the previous generation.
• A joint collaboration with NVIDIA optimized the pipeline, with the interaction network kernel improving 5.3x to over 10 billion ads per second and index matching exceeding 100 billion ads per second.
• The system co-locates index matching, scoring, and top-K selection on a single GPU, reducing network traffic by approximately 100x and eliminating memory bandwidth bottlenecks.

Technical Architecture

• The model transitions from a single user tower to an ensemble of specialized user towers (experts) to capture diverse and time-evolving user interests.
• An MoE attention-based gating network dynamically routes candidates to relevant experts, maintaining high expressiveness while minimizing compute cost.
• A fused GPU kernel executes the entire per-user-ad computation in one pass, keeping intermediate results in registers and shared memory to avoid High Bandwidth Memory (HBM) traffic.

Performance and Hardware

• The optimized gating kernel reaches approximately 85% of the GPU's peak HBM bandwidth on NVIDIA GH200 Grace Hopper Superchips.
• INT8 quantization provides up to 2x additional throughput over FP16 kernels with accuracy on par with 16-bit precision.
• Since 2025, the system has delivered a 2.4% increase in conversions on Facebook by matching ads across a larger candidate pool within the same latency budget.