Toss Improves CVR by 7.55% by Optimizing Recommendation TopK with Integer Programming

The Toss Shopping recommendation system had empirically determined the number of candidates (TopK) retrieved at the Retrieval stage. A high number of candidates increased Ranking costs and response times, and could include low-value candidates, necessitating efficient allocation.

The Toss Commerce Personalization Team defined this problem as a knapsack problem and applied Integer Programming. Under the constraint of limiting the total number of candidates to approximately 50% of the previous level, they calculated the model-specific TopK to maximize the sum of the expected purchase value (pICVR) for each retrieval model.

Results from a 7-day online A/B test showed that the Integer Programming experimental group saw a 7.55% increase in User CVR and a 10.13% increase in Order PU (orders per user) compared to the control group. Conversely, CTR decreased by 3.71%, suggesting that while click opportunities were reduced, the purchase conversion efficiency of the remaining clicks improved. Additionally, the number of unique exposed products decreased by 22.07%, leading to a reduction in diversity.

In subsequent experiments, Gumbel Weighted Sampling was applied, reducing feed duplication by 11.25% and increasing exposure diversity by 4.61%. Toss is currently considering personalized TopK optimization that assigns different TopK values per model based on user history and state.