Lightweight Ad Engagement Prediction with Two-Tower Models Served on GPU

Pinterest's ad lightweight ranking is a stage that quickly filters candidate ads before passing them to the more complex downstream ranking model. Here, the two-tower structure is retained, with the Pin (ad) tower generating embeddings in offline batches and the query (user) tower generating embeddings in real time; a score is computed by applying sigmoid to the dot product of the two embeddings.

Previously, all two-tower models were served on CPU, but in 2025 the first GPU-serving model for engagement prediction was launched. The new model shifted from the existing MTMD to a combination of MMOE and DCN, with each expert jointly using full-rank and low-rank DCN layers. Thanks to GPU serving, a more complex structure could be used while maintaining latency similar to the CPU baseline.

In terms of performance, offline loss dropped by 5~10% compared to the previous production CTR prediction model, and separating standard ad and shopping ad scenarios to train each on only its relevant data reduced loss by an additional 5~10%. This separation approach also boosted the offline model iteration speed by 2x.

The following optimizations were applied to improve training efficiency.

• Dataloader optimization: Configured GPU prefetch so that the GPU processes batch i while batch i+1 is being prepared, and increased the number of worker threads by leveraging the 1TB of CPU memory on p4d instances.
• Model code efficiency: Avoided expensive zero allocation on CPU by handling it directly on GPU, and used fused kernels instead of multiple individual kernels.
• Training configuration improvements: Introduced BF16 precision and increased batch size to improve memory utilization.

For training, the prediction score of the downstream ranking model was used as the label, and KL divergence between the label and the prediction was used as the loss. Evaluation targets were auction winners and auction candidates, and loss reduction was confirmed in both offline and online settings. In online experiments, CPC decreased and CTR increased, showing that improvements in GPU infrastructure and model architecture worked together to produce results.