The Warmth Hidden Behind Star Ratings: Building Differentiated Recommendations with LLMs

Due to the 'star rating inflation' phenomenon, where most restaurant reviews cluster around 5 stars, users cannot trust a store based on star ratings alone and are forced to go through the inconvenience of manually verifying review texts one by one.

To solve this, an LLM-as-a-Judge approach was introduced to precisely quantify the 'warmth of sincerity' contained in review texts. Rather than simple positive/negative classification, the LLM comprehensively analyzes subtle modifiers, intent to revisit, and specific taste descriptions to determine the authenticity of a review.

To address the cost problem of processing tens of millions of reviews via a paid API, a Knowledge Distillation strategy was used. Based on high-quality labeled data generated by a high-performance Teacher Model (LLM), a Student Model (ELECTRA) pre-trained on in-house data was trained, securing inference speed and performance optimized for the production service environment.

Ultimately, a bonus-point approach was adopted, adding the AI score on top of the existing star rating. By applying Bayesian Smoothing, distortion due to data volume was prevented, resulting in a sophisticated recommendation system that maintains the existing hierarchy of star ratings while also reflecting qualitative differences in reviews.