Kakao Presents Three Key Criteria for Selecting Collaborative Filtering Models

Kakao's recommendation team emphasizes that when developing Collaborative Filtering (CF) models, the 'problem to be solved' must be defined first, rather than the model itself. This is because the suitable model varies depending on factors such as the availability of sufficient feedback data, the size of recommendation slots, and whether the goal is prediction accuracy or ranking optimization.

Model Selection Based on Feedback Data Characteristics

CF models utilize user-item interaction data. The approach differs between cases with rich Explicit Feedback (e.g., ratings) and cases with only Implicit Feedback (e.g., clicks, dwell time). In particular, Implicit Feedback involves uncertainty regarding whether a user dislikes an item they did not consume or if it was simply not exposed to them. To address this, the Alternating Least Squares (ALS) model introduces the concept of Confidence to reflect the uncertainty of unobserved data in training.

Learning Framework Based on Optimization Goals

The model changes depending on whether the goal is score prediction or ranking optimization. When ranking performance is critical, instead of a general Point-wise loss function, a Pair-wise learning framework such as Bayesian Personalized Ranking (BPR) is advantageous. BPR provides superior ranking performance compared to existing techniques by directly optimizing AUC through comparing positive/negative item pairs.

Addressing Data Bias and Feedback Loops

Training data for recommendation systems consists of observational data rather than controlled experiments, leading to exposure bias (Selection Bias) and feedback loop issues. This can result in offline evaluations (e.g., nDCG) that favor existing models. Since random data collection is limited due to degraded user experience, research on Debiasing methodologies for fair evaluation on biased data is ongoing.