Fortress: A Case Study on Stabilizing Search and Recommendation Systems through Temporal Data Augmentation and Feature Pruning

Prediction models in search and recommendation systems suffer from Temporal Instability, a problem in which specific input Features cause fluctuations in output scores. This instability undermines the model's reliability and negatively affects decision-making, especially in multi-stage systems where consistent predictions are essential.

To address this, the proposed Fortress is a general-purpose framework that simultaneously improves model stability and accuracy. This framework uses Feature Pruning, a method that identifies and removes Features that cause prediction scores to become inconsistent over time.

The core mechanisms of Fortress are as follows:

• Temporal Data Augmentation: Tests model robustness by leveraging temporally split datasets with historical snapshots.
• Feature Pruning: Ensures model consistency by identifying and removing specific Features that cause prediction fluctuations.
• Stability & Accuracy Optimization: Optimizes the model to maintain stable predictions even in data environments that change over time.