AI Briefing
KO

Improving the Place Review Tagging System to Achieve a 67% Update Compression Rate

·2023.11.06 16:06

Key point

By separating tagging logic from storage and adding a write buffer, updates were reduced by 67%.

1 / 2

Details

The existing structure, which stored analysis results extracted from review images and text directly as tags, was vulnerable to business logic changes, difficult to integrate with operational tools, and caused DB updates to occur far too frequently.

There were two key improvements.

  • Store the raw analysis values in rawtags to reduce re-analysis costs, and separate the tagging business logic into its own module
  • Consolidate all tags into a tags collection to enable operational lookup and editing, and separate operator-entered values with valueByAdmin
  • Introduce a write buffer called updatabletargets to batch updates for the same target, then perform a bulk update every 20 minutes

The flow works as follows: the first Kafka Consumer, which detects a review insert, calls the analysis API and stores the rawtag. The second Consumer generates tags using logic specific to each rawtag type. The third Consumer records the target to be updated in updatabletargets, and Airflow scheduling applies the accumulated tags all at once.

As a result, when applied to the current three image tagging types—mood recognition, image scoring, and Object Detection—a 66.7% update compression rate was confirmed compared to before. In the example logs, KR image tagging dropped from 10274 cases to 3425 cases, and JP dropped from 948 cases to 316 cases. Propagation from images to reviews was also compressed to 47.1% for KR and 31.9% for JP.

On the operational side, consolidating tags in one place made filtering, lookup, and editing possible, and reduced the likelihood that operator-entered values would be overwritten and lost by analysis values. Additionally, manually edited data is now sent to the Place Data Lakehouse, connecting it to the ML retraining pipeline.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.