Toss Improves CVR by 0.91% by Resolving Training-Serving Skew
·2026.11.17 10:00
Key point
Applied serving partition logging and blue/green deployment to reduce feature discrepancies between training and serving.
Details
- To resolve the Training-Serving Skew issue in Toss's recommendation system, the team logged the actual partition information used during serving and improved the training data generation process.
- Online experiment results showed a 0.86% decrease in CTR compared to the existing model, but a 0.91% increase in CVR, with no significant changes in revenue-related guardrail metrics.
- Instead of logging the entire feature values, the team chose to log only the used partitions, reducing daily log storage costs that reached tens of TB.
- Blue/green deployment was applied to prevent inconsistencies between feature values and partition information during data updates, and the uploader logic was modified to immediately delete data excluded from the aggregation target.
- Serving partitions could be linked to approximately 89% of user features and approximately 94% of product features, while cross features reproduced actual serving values in approximately 75% of the analysis target.
- The lookup rate for cross features with residual past values decreased from 14.4% to 6.0%, reducing the scope in which the model used noisy features.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.