How Should Long-Term User Modeling Change in a Local Super App

Recent behavior alone struggles to fully capture recurring interests, preferences that span multiple verticals, and the selection bias created by recommendations themselves. However, simply extending the history increases latency and infrastructure complexity, so long-period data, a design that can learn well, and a servable architecture all needed to be solved together.

The solution was a structure where a separate user encoder trains and infers on long-term history offline, and this is reused as a common user feature by home feed ranking, candidate models, and ad ranking. This way, downstream models don't need to hold long-term history directly, and the user encoder can independently scale up data and compute.

The user encoder was built as a two-tower structure. The User Tower feeds the user's action sequence into a Causal Transformer to create a user embedding, and the Item Tower embeds item features with an MLP, trained with an InfoNCE loss that predicts the next action item. Clicks and conversions from multiple verticals such as secondhand trading, part-time jobs, real estate, and used cars were all used, and it was trained on about 150x more logs—tens of billions—than the existing home feed two-tower candidate model.

For item representation, Item ID embedding was compared against content embedding. The ID-based approach had significant cold-start and GPU memory issues, with over 99% of all parameters occupied by the embedding table, making it hard to scale up the Transformer. In contrast, using LLM-based content embedding allowed even new items to be represented with just metadata, and removing the embedding table made it possible to scale the Transformer up by 1,000x.

However, attaching content embeddings for hundreds of millions of items to training was itself a major problem. This was solved by reading only the needed portions from disk using memmap, and handling the item ID → location mapping with bbhash (minimal perfect hash), reducing memory by about 97% compared to a Python dict.

The most interesting part was contrastive learning in a location-based service. At Danggeun, since over 86% of transactions occur within a 5km radius, about 98% of a random batch were impossible negatives that the user could never have seen in the first place. In other words, the model was wasting its learning signal on distinguishing exposure feasibility rather than actual taste.

To address this, Region-Constrained Batch Sampling (RCBS) was applied, which changes only the batch composition while keeping the model and loss unchanged. By grouping users from the same region into a batch, the proportion of feasible negatives was increased, and impossible negatives dropped from 98% → 30%. Since the model now had to distinguish between similar items within the same neighborhood, feasible negatives became harder negatives, and this was confirmed during evaluation, where RCBS-Eval scored lower than Random-Eval.

Masking impossible negatives within a batch makes the effective batch size too small, and hard negative mining requires separately considering per-user feasibility, adding complexity. RCBS, on the other hand, was simpler and more efficient since it could naturally create harder negatives just by changing batch sampling.

For downstream application, the team tried concatenating the user embedding after a projection layer for home feed/ad ranking, and for candidates (retrieval), even tried generating the candidate pool using only the user embedding. In particular, the configuration using only the user embedding for the candidate pool performed best, and having candidates of different natures had the effect of increasing diversity.

The user embedding update cycle was also tested. Offline, the differences between fixed, 24-hour, and 12-hour updates were not large, but online, periodic inference was better than fixed, and shorter cycles were better. Considering the balance between cost and performance, a 24-hour cycle was chosen, and only the necessary users were re-embedded via a beam pipeline that runs GPU inference on GCP Dataflow.

In offline pretraining, RCBS-Train(fine) showed improvements of Recall@10 +49% (Random-Eval) and +70% (RCBS-Eval fine) over Random-Train. RCBS-Train(fine) embeddings were consistently better in downstream tasks as well, and in the online A/B test, multiple metrics improved together, including clicks, impressions, DAV, app dwell time, and ad revenue.