Pinterest Unveils 3-Tower Engagement Co-Train Model: Achieves 1% CTR Lift and 30ms Latency

The Pinterest Ads Lightweight Ranking team has unveiled the 3-Tower Engagement Co-Train model. This model maintains the fast inference speed of the existing 2-Tower architecture while improving prediction accuracy by adding a Cross Tower and an Inter Module to capture complex interactions between user sequences and candidate ads.

Architecture and Training Strategy

The model simultaneously predicts pCTR, pGCTR30 (valid clicks lasting over 30 seconds), and pOCTR (external clicks). By splitting Query/Pin embeddings into task-specific segments, it runs a 64-dimensional 2-Tower for CTR in parallel with a 3-Tower that uses the full segments. gCTR30 and oCTR share the full embeddings.

• Cross Tower: Computes attention between user sequences (organic/ads engagement, search history) and candidate features (advertiser ID, GraphSAGE, etc.).
• Inter Module: Takes Query, Pin, and Cross embeddings as input and optimizes them using a Shared-bottom MLP structure to balance latency and performance.
• Co-Train Loss: Trained via a weighted sum of the main 3-Tower loss and the auxiliary 2-Tower loss, assigning higher weight to the main CTR.

Serving Optimization and 2-Stage Pipeline

Under the constraint of scoring over 200K candidates for P99, P99 latency was reduced from over 200ms to approximately 30ms. This was achieved by lowering precision from FP32 to BF16 and moving Pin preprocessing to offline caching.

To maximize serving efficiency, a 2-Stage pipeline was introduced.

1. Stage 1: Quickly scores all candidates using the 2-Tower head of the Co-Train model.
2. Stage 2: Rescores only the top 40K candidates by utility using the 3-Tower model.

Results

In offline experiments, the loss of the 3-Tower model decreased by approximately 30% compared to the existing production model. Online A/B test results showed that CTR, gCTR30, and oCTR increased by approximately 1%, while CPC decreased by approximately 1%. Although GPU spending increased slightly, the gap between predicted CTR and actual CTR was reduced, improving ad efficiency.